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Praise for Bulletproof Problem Solving 


“The Bulletproof Problem Solving approach acknowledges the reality many 
environmentalists face today: this is hard work. Conn and McLean's guide 
makes it a little bit easier.” 


-Mark R. Tercek, CEO of The Nature Conservancy and author of Nature’s 
Fortune 


“Conn and McLean have distilled their matchless experience in attacking 
challenges of every scale and level of complexity into this virtual war-room of a 
book, creating an indispensable resource for the 21st century problem-solvers 
upon whom our future depends. A must-read for all aspiring change agents!” 


-Sally Osberg, retired CEO of the Skoll Foundation, co-author of Getting 
Beyond Better: How Social Entrepreneurship Works 


“Navigating ambiguity and solving complex problems creatively is the truth 
test for humans to complement rather than substitute the artificial intelli- 
gence of computers. Without much better approaches to teach those skills, 
our schools risk preparing second class robots rather than first class humans. 
Rob McLean and Charles Conn show that this can be done and provide an 
intuitive roadmap for how to do this, with lots of real-world examples that 
make it fun.” 


-Andreas Schleicher, Director for the Directorate of Education and 
Skills, OECD 


“Great strategic problem solving is an essential tool, one whose value is only 
going up. Bulletproof provides the secret sauce behind the McKinsey frame- 
work to help structure and guide the problem-solving process. | want to hire 
people who understand this approach.” 


-Barry Nalebuff, Milton Steinbach Professor, Yale School of Management 
and cofounder, Honest Tea 


“The old paradigm of strategy departments and planning cycles has been 
overthrown by agile and rapid team-based problem solving, providing better 
solutions and better organization alignment to implement. This book, written 
by two of the smartest people | know, provides the needed blueprint for how 
build these world-beating problem solving teams.” 


-Mehrdad Baghai, Chair of Alchemy Growth and author of As One 


“The world has never been in more need of extraordinary problem solvers-in 
business and every other walk of life. Rob McLean and Charles Conn power- 
fully demonstrate that problem solving is a structured process that can be 
learned and applied to the benefit of everybody. Their book is such an impor- 
tant contribution to the resolution of our biggest problem solving challenges.” 


-Nick Lovegrove, Professor of the Practice, Georgetown University and 
author of The Mosaic Principle 
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Foreword 


Bulletproof. At McKinsey there is no greater compliment than to 
have your reputation as a problem solver described as “bullet- 
proof.” While it takes many skills and types of intelligence to make 
a modern consulting firm work, the cornerstone capability is always 
creative problem solving. 


The importance of great problem solving has only grown as the 
pace of economic and technological change has accelerated in 
recent years—and the scope and complexity of the problems we 
need to address increases alongside it. Today we are just as likely 
to be hired to help a country public health system prepare for the 
next Ebola outbreak as to develop a digital marketing strategy for a 
new consumer product. As ever more data becomes available, the 
bar on the quality of thinking rises. We need bulletproof problem 
solvers. 


Whether you work in industry, the nonprofit sector, or government, 
there is no way to anticipate and plan for the new structures and 
operating rules that are unfolding. Nor is simply accelerating and 
adapting traditional, domain-oriented, training approaches suffi- 
cient. The only way to successfully navigate this level of change 
is to be a fluid and creative problem solver. That's why the World 
Economic Forum labeled complex problem solving its number one 
skill for the twenty-first century. Organizations everywhere are look- 
ing for this capability in their talent recruiting above all else. 


Foreword 


What is perhaps surprising is that a disciplined, comprehensive 
approach to problem solving isn't taught in schools or universities. 
It is absent from most curricula even in many business schools. You 
can see elements in things like root-cause analysis or the current 
vogue for agile teams and design thinking, but they don’t go far 
enough. This book introduces the systematic process for problem 
solving that has been missing, a version of the time-tested method- 
ology we have used for many years in McKinsey. 


The seven-step method Charles and Rob demonstrate here is 
transparent and straightforward. It doesn't require specialist skills 
or fancy mathematical talent—though the authors do show when 
more sophisticated analytic techniques can be valuable, and why 
they are often more accessible than you think. It is iterative and 
flexible; it can be applied quickly to get rough-cut answers, and 
more slowly to fine-tune nuanced answers. It shows how to fight 
the human biases in decision making that we have learned so much 
about in recent years. And it works on nearly any kind of problem, 
from personal life decisions, to business and nonprofit questions, 
to the biggest policy challenges facing society. 


As a longtime runner, | was especially drawn to Rob’s analysis 
of whether or not to have knee surgery. | was also impressed by 
the straightforward analysis that can help voters consider their 
response to complicated policy decisions in areas like fisheries and 
educational funding. | naturally enjoyed reading the cases cover- 
ing business strategy or enhancing profitability. And while there 
are some genuinely intractable social and environmental problems, 
this methodology can still shine light on solution paths to even the 
trickiest challenges, including fighting climate change and obesity. 


You couldn't ask for more qualified authors to write a book of this 
kind. Charles drafted the original internal McKinsey presentation 
on problem solving, 7 Easy Steps to Bulletproof Problem Solving, 
one of our most requested professional development documents 
ever, when we were young consultants in Toronto. | have known Rob 
for more than 35 years, starting with a project we did together on 
how to leverage the time of the CEO of Australia’s largest company. 


Foreword 


During their time at McKinsey, Rob and Charles collaborated with 
other colleagues to develop the horizons approach to growth strat- 
egy that we still use today. After they left the firm, | enjoyed watch- 
ing them both continue to apply their problem solving method 
as entrepreneurs and as change makers in the nonprofit sector. In 
recent years | have had a front-row seat as Charles brought this 
distinctive mindset to strategy development and transformation at 
the Rhodes Trust. 


Problem solving is the core skill for the twenty-first century. Now, 
finally, we have a guide to doing it right that any of us can follow. 


Dominic Barton 


Managing Director (Retired), 
McKinsey & Company 
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Introduction 


Problem Solving for the 
Challenges of the 
Twenty-First Century 


Great problem solving has never been more important for business 
and society. The problems facing humankind are larger, more com- 
plex, and moving faster than ever before. Previous approaches to 
training for careers are now outmoded as change in technologies 
and business models accelerates. Learning how to define a prob- 
lem, creatively break it into manageable parts, and systematically 
work toward a solution has become the core skill for the twenty-first 
century workforce, the only way to keep up. But how problem solv- 
ing is taught in our schools, universities, businesses, and organiza- 
tions is coming up short. We need a new approach. 


Let's start with a definition: 


Problem solving is decision making when there is complexity and 
uncertainty that rules out obvious answers, and where there are 
consequences that make the work to get good answers worth it 


We all know the consequences of poor problem solving can be 
costly to business and communities, human health, and the envi- 
ronment. This book introduces a long-tested and systematic 
approach that can be taught to anyone who wants to become a 
better problem solver, from corporate strategists to nonprofit 
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workers. This powerful framework, Bulletproof Problem Solving, is an 
approach we learned and helped develop at McKinsey & Company, 
the global consulting firm. This seven-step process hasn't been 
shared widely outside McKinsey until now. It can be used by indi- 
viduals, teams, executives, government policy makers, and social 
entrepreneurs—anyone with a complex and uncertain problem of 
consequence. This systematic approach to problem solving could 
help you get a great job, make you more effective in your work, 
make your role as a citizen more fulfilling, and even make your per- 
sonal life work better. These are big claims, but we know it works. 


Problem Solving Capability 


This new era of focus on creative problem solving has been ushered 
in by massive disruption of the old order in business and society. New 
business models are rapidly emerging from revolutionary Internet, 
machine learning, and bioscience technologies that threaten the 
status quo in every field. New rules are being written for conduct- 
ing business and dealing with social and environmental challenges. 
Succeeding requires complex problem solving skills as never before. 
If you're a product manager who faces disruptive competition, you 
need to have a game plan and command of resources to overcome 
competition. You will only get resources if you make a persuasive case 
based on hypotheses about a winning plan, accompanied by analysis 
to support the key propositions. If you're a nonprofit leader of a team 
dealing with communities facing generational disadvantage who has 
seen new initiatives come and go, you have to be able to articulate a 
theory of change that links issues with interventions and outcomes if 
you want support from the board of your organization. 


As organizations seek to become clever and agile to address this new 
world, they take on the persona of problem solving organizations—a 
drive to be working on the right problems, addressing root causes, 
engaging teams around short duration work plans, and allocating 
responsibilities and timelines with accountability. Over the course 
of our careers, we have seen the focus of organizational capability 
aspirations shift through distinct eras: from strategy to execution to 
complex problem solving. 
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The 70s and 80s were characterized by intense interest in strategy 
development. That was displaced by an era from the 90s onward 
that focused on execution, including deep attention to getting 
things done, as exemplified by the book Execution by Ram Charan 
and Larry Bossidy, and a number of books on business process rede- 
sign.' However, a ruthless focus on execution assumes you have 
strategic direction right and can adapt to new competition, fre- 
quently from outside your industry. This can no longer be assumed. 


MANAGERIAL SKILLS EVOLUTION 


ERA Strategy ener Complex Problem 
Solving 
TIME PERIOD 70s - 80s 90s - 2015 2015 onward 
CHALLENGES Where and how Getting things Agile and creative 
to compete done problem solving 


As this new era of the problem solving organization takes hold, we 
expect it will trigger even more interest in how teams go about 
sharpening complex problem solving and critical thinking skills— 
what is called mental muscle by the authors of The Mathematical 
Corporation. The other side of the equation is the increasing 
importance of machine learning and artificial intelligence in 
addressing fast-changing systems. Problem solving will increas- 
ingly utilize advances in machine learning to predict patterns in 
consumer behavior, disease, credit risk, and other complex phe- 
nomena, termed machine muscle. 


To meet the challenges of the twenty-first century, mental muscle 
and machine muscle have to work together. Machine learning frees 


‘Larry Bossidy and Ram Charan, Execution: The Discipline of Getting Things Done 
(Random House, 2008). 


? Josh Sullivan and Angela Zutavern, The Mathematical Corporation: Where Machine 
Intelligence and Human Ingenuity Achieve the Impossible (Public Affairs, 2017). 
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human problem solvers from computational drudgery and amplifies 
the pattern recognition required for faster organizational response 
to external challenges. For this partnership to work, twenty-first cen- 
tury organizations need staff who are quick on their feet, who learn 
new skills quickly, and who attack emerging problems with confi- 
dence. The World Economic Forum in its Future of Jobs Report? 
placed complex problem solving at #1 in its top 10 skills for jobs in 
2020. Here is their list of important skills that employers are seeking: 


TOP Ne SSIES Is Lee 


Complex Problem Solving 


People Management 


Coordinating with Others 
Emotional Intelligence 
Judgment and Decision Making 


Service Orientation 


2 OM DM SWN 


Negotiation 


= 
(a) 


Cognitive Flexibility 


It is becoming very clear that job growth is focused in areas where 
tasks are nonroutine and cognitive, versus routine and manual. The 
intersection of nonroutine tasks and cognitive ability is the heart- 
land of complex problem solving. The authors of a recent McKinsey 
Quarterly article made the point that “more and more positions 


3 Future of Jobs: Employment, Skills and Workforce Strategy for the Fourth Indus- 
trial Revolution (World Economic Forum, 2016). 
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require employees with deeper expertise, more independent judg- 
ment, and better problem solving skills.”* We are already seeing 
that many organizations place a premium on analytic skills and 
problem solving and make it the essential criterion to be hired. 
Commentator David Brooks of the New York Times takes this con- 
clusion even further when he says, “It doesn’t matter if you are work- 
ing in the cafeteria or the inspection line of a plant, companies will 
only hire people who can see problems and organize responses.”° 


Education Gaps 


If creative problem solving is the critical twenty-first century skill, what 
are schools and universities doing to develop these skills in students? 
Not enough. It remains early days in codifying and disseminating 
problem solving best practices in educational institutions. Andreas 
Schleicher, Director of Education and Skills and Special Advisor to 
the Secretary General of the OECD, explains the need for develop- 
ing problem solving skills in students this way: “Put simply, the world 
no longer rewards people just for what they know—Google knows 
everything—but for what they can do with what they know. Problem 
solving is at the heart of this, the capacity of an individual to engage 
in cognitive processing to understand and resolve problem situa- 
tions where a method of solution is not immediately obvious. ”ć 


The OECD Program for International Student Assessment (PISA) 
started testing individual problem solving skills in 2012 and added 
collaborative problem solving skills in the 2015 assessments. One of 
the interesting early findings is that to teach students to become bet- 
ter problem solvers involves other capabilities than simply teaching 
reading, mathematics, and science literacy well. Capabilities such as 
creativity, logic, and reasoning are essential contributors to students 
becoming better problem solvers. That is what this book is about. 


“Boris Ewenstein, Bryan Hancock, and Asmus Komm, “Ahead of the Curve: The 
Future of Performance Management,” McKinsey Quarterly, May 2016. 


5 David Brooks, “Everyone a Changemaker,” New York Times, February 18, 2018. 


éBeno Csapo and Joachim Funke (eds.), The Nature of Problem Solving: Using 
Research to Inspire 21st Century Learning. (OECD Publishing, 2017). 
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Universities and colleges are being challenged to demonstrate that 
their graduates have developed problem solving skills to prepare 
them for the demands of the workplace. One method of evaluating 
whether over a college degree there is improvement in critical think- 
ing is the CLA+ test (Collegiate Learning Assessment plus) developed 
by the nonprofit Council for Aid to Education (CAE). The Wall Street 
Journal reported in 2017 that of the 200 colleges that apply the test 
“a majority of colleges that took the CLA+ made measurable pro- 
gress in critical thinking”—although some well-respected colleges 
didn't show much difference between incoming freshmen scores and 
those of seniors.’ Effective university approaches to develop critical 
thinking and problem solving range from analyzing classic poems 
like Beowulf, to teaching logic structures, and setting practical group 
projects that require demonstration of problem solving abilities. 
What we glean from the article and college practices generally is an 
awakening of interest in student problem solving, and expectations 
that problem solving will be enhanced over the course of a degree 
program. But we have not seen a common framework or process 
emerge yet. 


The Seven-Steps Process 


The heart of the book is a seven-step framework for creative prob- 
lem solving, Bulletproof Problem Solving, starting with these critical 
questions: 


1. How do you define a problem in a precise way to meet the 
decision maker's needs? 


2. How do you disaggregate the issues and develop hypotheses 
to be explored? 


3. How do you prioritize what to do and what not to do? 
4. How do you develop a workplan and assign analytical tasks? 


5. How do you decide on the fact gathering and analysis to resolve 
the issues, while avoiding cognitive biases? 


’Douglas Belkin, “Exclusive Test Data: Many Colleges Fail to Improve Critical- 
Thinking Skills,” Wall Street Journal, June 5, 2017. 
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6. How do you go about synthesizing the findings to highlight 
insights? 


7. How do you communicate them in a compelling way? 


aye SEVEN STEp, 


DEFINE 
y uE PROBLEM a 


In the book we take you through the seven steps in a way that builds 
understanding and mastery through examples. We highlight a vari- 
ety of analytic tools available to aid this process, from clever heuris- 
tics, analytic short cuts, and back-of-the-envelope calculations, to 
sophisticated tools such as game theory, regression analysis, and 
machine learning. We also show how common cognitive biases can 
be addressed as part of the problem solving process. 


The final two chapters explicitly deal with how you solve prob- 
lems when uncertainty is high and interdependencies or systems 
effects are significant. We believe that even the so-called “wicked 
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problems” of society can be tackled, such as obesity and environmen- 
tal degradation. These are tough problems that have multiple causes, 
are affected by externalities, require human behavioral change, and 
have some solutions that may bring unintended consequences. These 
chapters are for people dealing with advanced problem solving situ- 
ations, but the cases are fascinating reading for anyone interested in 
the major issues business and society needs to address. 


High Stakes 


Good problem solving has the potential to save lives and change 
the fortunes of companies, nonprofits, and governments. On the 
other hand, mistakes in problem solving are often very costly and 
sometimes can cause great harm, as we saw in the Space Shuttle 
Challenger disaster. 


THE SPACE SHUTTLE CHALLENGEK 
A TRAGIC ERROR IN ANALYSIS 


On January 28, 1986 the Space Shuttle Challenger exploded over Cape Canaveral, Florida 
73 seconds after lift-off into its tenth mission. The consequences for the families of the 
astronauts watching below, the NASA aerospace engineers, mission control workers, and the 
American space program itself were tragic and catastrophic. For those of us who watched 
this tragedy unfold on televisions around the world, we could only imagine the horror and 
devastation. The Challenger exploded because of a failure of its O-rings — small rubber rings 
that prevent hot gases from leaking. These conclusions emerged from the Rogers Commission 
and the work of physicist Richard Feynman 


While the Challenger case has been studied over many years, it remains relevant to this book 
That's because it wasn't just an O-ring failure, as most had come to accept, but a problem- 
solving failure, and that aspect of the case is little understood. The O-ring failure was 
brought on by Mother Nature herself, a very unusual drop in temperature in Florida the day 
of the launch — 21°F colder than the coldest previous launch at 53°F. At this dramatically 
colder temperature, the rubber in the O-rings failed to expand 


The error lay in an incomplete and flawed analysis linking failed O-rings and temperature at 
launch. It was only when the full data set of successful and unsuccessful launches is examined 
that the risks of failure at such a low temperature becomes clear. NASA put the chance of 
failure at 1 in 100,000, whereas Feynman argued it was more like 1 in 100. More recent re- 
analysis by Bayesian researchers put the probability of failure at the launch temperature 
close to certainty. We discuss the Challenger case in more detail in Chapter 6 
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Pitfalls and Common Mistakes 


When we listen to people describe their approach to prob- 
lem solving, they invariably identify one step they feel they do 
well. Some will confidently describe their approach to problem 
definition as SMART (specific, measurable, actionable, relevant, 
and time frame); others will cite their knowledge of inductive 
and deductive logic; some will point to their workplans bringing 
accountability to team processes; many will point to their ability 
to do fact gathering and analysis; and a few will mention the way 
they use the pyramid principle to write a persuasive document 
with a governing thought. But we see very few who say they do 
all the above, coupled with a way to cleave problems and address 
bias. To do good problem solving, you have to do all the steps 
in concert. This is what is so powerful and distinctive about the 
seven-steps process. 


Despite increasing focus on problem solving in schools and uni- 
versities, businesses, and nonprofits, we find that there is con- 
fusion about what good problem solving entails, There are a 
number of pitfalls and common mistakes that many make. These 
include: 


1. Weak problem statements. Too many problem statements lack 
specificity, clarity around decision-maker criteria and constraints, 
an indication of action that will occur if the problem is solved, or 
a time frame or required level of accuracy for solving the prob- 
lem. Rushing into analysis with a vague problem statement is a 
clear formula for long hours and frustrated clients. 


2. Asserting the answer. The assertion is often based on experi- 
ence or analogy ("I’ve seen this before”), without testing to see if 
that solution is really a good fit for the problem at hand. Answers 
like this are corrupted by availability bias (drawing only on facts 
at hand), anchoring bias (selecting a numerical range you have 
seen already), or confirmation bias (seeing only data that aligns 
with your prejudices). 
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3. Failure to disaggregate the problem. We see few problems 
that can ever be solved without disaggregation into compo- 
nent parts. A team looking at the burden of asthma in Sydney 
got the critical insight into the problem only when they broke 
it down along the lines of incidence and severity. In Western 
Sydney the incidence of asthma was only 10% higher than 
Northern Sydney, but deaths and hospitalization were 54-65% 
greater. The team was familiar with research that linked asthma 
with socioeconomic status and tree cover. It turns out that 
socioeconomic status is significantly lower in Western Sydney, 
tree cover is about half Northern Sydney, and daily maximum 
particulate matter (PM 2.5) is 50% higher. By finding the right 
cleaving point to disaggregate the problem, the team was able 
to focus on the crux of the issue. This led to them proposing 
an innovative approach to address respiratory health through 
natural solutions, such as increasing tree cover to absorb par- 
ticulate matter. 


4. Neglecting team structure and norms. Our experiences in team 
problem solving in McKinsey and other organizations highlight 
the importance of a diversity of experience and divergent views 
in the group, having people who are open-minded, a group 
dynamic that can be either competitive or collaborative, and 
training and team processes to reduce the impact of biases. This 
has been underscored by recent work on forecasting.® Execu- 
tives rank reducing decision bias as their number one aspiration 
for improving performance.’ For example, a food products com- 
pany Rob was serving was trying to exit a loss-making business. 
They could have drawn a line under the losses if they took an 
offer to exit when they had lost $125 million. But they would only 
accept offers to recover accounting book value (a measure of 
the original cost). Their loss aversion, a form of sunk-cost bias, 


8Philip Tetlock and Dan Gardner, Superforecasting: The Art and Science of Predic- 
tion (Random House, 2015). 

? Tobias Baer, Sven Hellistag, and Hamid Samandari, “The Business Logic in Debi- 
asing,” McKinsey Latest Thinking, May 2017. 
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meant that several years later they finally exited with losses in 
excess of $500 million! Groupthink amongst a team of manag- 
ers with similar backgrounds and traditional hierarchy made it 
hard for them see the real alternatives clearly; this is a common 
problem in business. 


5. Incomplete analytic tool set. Some issues can be resolved with 
back of the envelope calculations. Others demand time and 
sophisticated techniques. For example, sometimes no amount 
of regression analysis is a substitute for a well-designed, real- 
world experiment that allows variables to be controlled and a 
valid counterfactual examined. Other times analysis fails because 
teams don't have the right tools. We often see overbidding for 
assets where teams use past earnings multiples rather than the 
present value of future cash flows. We also see underbidding for 
assets where development options and abandonment options, 
concepts akin to financial options, are not explicitly valued. How 
BHP, an Australian resource company, addressed these issues is 
developed in Chapter 8. 


6. Failing to link conclusions with a storyline for action. Analyti- 
cally oriented teams often say, "We're done” when the analysis 
is complete, but without thinking about how to synthesize and 
communicate complex concepts to diverse audiences. For exam- 
ple, ecologists have pointed to the aspects of nature and urban 
green spaces that promote human well-being. The message 
has frequently been lost in the technical language of ecosystem 
services—that is, in describing the important role that bees play 
in pollination, that trees play in absorbing particulate matter, 
or water catchments play in providing drinking water. The story 
becomes so much more compelling when, in the case of air pol- 
lution, it has been linked to human respiratory health improve- 
ments in asthma and cardiovascular disease. In this case, by 
completing the circle and finding a way to develop a compelling 


10 Planting Healthy Air (The Nature Conservancy, 2016). 
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storyline that links back to the “hook” of human health makes all 
the difference in capturing an audience and compelling action. 


7. Treating the problem solving process as one-off rather than an 
iterative one. Rarely is a problem solved once and for all. Prob- 
lems we will discuss often have a messiness about them that 
takes you back and forth between hypotheses, analysis, and con- 
clusions, each time deepening your understanding. We provide 
examples to show it is okay and worthwhile to have second and 
third iterations of issue trees as your understanding of a prob- 
lem changes. 


What's in Store? 


This is a how-to book. We work through 30 real-world examples, 
employing a highly visual logic-tree approach, with more than 
90 graphics. These are drawn from our experience and honed over 
an intensive summer of research with a team of Rhodes Scholars in 
Oxford. They include problems as diverse as the supply of nurses 
in the San Francisco Bay Area, to capital investment decisions in 
an Australian mining company, to reduction of the spread of HIV 
in India, to air pollution and public health in London, to competi- 
tive dynamics in the hardware home-center industry, and even to 
approaches to address climate change. The insights in some cases 
are novel, in other cases counterintuitive. The real-world exam- 
ples behind the cases have created value amounting to billions of 
dollars, saved hundreds of thousands of lives, and improved the 
future for endangered species like salmon. 


If you want to become a better problem solver, we show how you 
can do so with only a modest amount of structure and numeric abil- 
ity. Individuals make decisions that have lifetime consequences— 
such as career choice, where to live, their savings plan, or elective 
surgery—often without due consideration. These are among the 
examples we walk you through in the book to illustrate the value of 
a structured process to improve your prospects of better outcomes 
in your own life. 


PROBLEM SOLVING CASES 


INDIVIDUAL 


» Should | put solar panels on my roof? 
» Should | support the school bond? 

» Where to live 

»ls where | live affecting my health? 

» Should | have a knee arthroscopye 


ORGANIZATIONS 


»Pricing decision at Truckgear 

» Competitor analysis-Home Depot vs. Hechinger 

» Drones to the rescue/shark spotting with 
machine learning 

» Airport capacity 

» Growth strategy at J&J 

» Drone company staircase 

» Should we defend our IP in courte 

» Bias in mineral exploration 


CITIZEN / POLICY 


» Protecting salmon in the Pacific Northwest 
»HIV in India 
»How to tackle climate change 


Introduction 


»How to judge contested characters in 
history 

» Will | outlive my savings? 

»What career should | choose? 

» Where to serve in tennis 


» Predicting sleep apnea with machine learning 

» Bus routing with machine learning 

» Classifying heart attack risks in hospitals 

» Oil refinery strategy communication 

» Electronic Arts: AIB testing 

» Crowdsourcing problem solving in organizations 
» Supply of quality nurses in Bay Area 

»How to make long-term resource investments 
»Root cause of market share loss 


» Can obesity be reduced? 
» Challenger Space Shuttle disaster 
»How to reduce overfishing 


As citizens we have a desire to understand issues of the day more 
clearly and to be able to make a contribution to resolving them. 
There is a temptation to say, “That issue is way too complex or 
political for me to add a perspective.” We hope to change your 
mind about that. There are few bigger problems on the planet than 
climate change, obesity, reducing the spread of infectious disease, 
and the protection of species, and we demonstrate how to tackle 
problems also at this societal scale. 


XXV 


xxvi 


Introduction 


For college students and graduates in analytical roles we hope this 
book will become an important resource for you—a comprehensive 
suite of tools and approaches that can make you a better problem 
solver, one you will return to again and again. For managers we set 
out how to evaluate your competitor's performance, decide where 
and how to compete, and develop a strategy in uncertain and com- 
plex settings. 


Our aim is simple: to enable readers to become better problem 
solvers in all aspects of their lives. You don’t need post-graduate 
training to be an effective problem solver. You do need to be pre- 
pared to work through a process and develop cases of your own 
where you can try-test-learn the framework. This quote from Nobel 
Laureate Herb Simon captures much of what we set out to do in 
the book: “Solving a problem simply means representing it so as to 
make the solution transparent.” 


11 Herbert Simon, The Sciences of the Artificial (MIT Press, 1968). 
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Learn the Bulletproof Problem Solving Approach 


In the 1980s, when Charles was at business school, he wanted to 
understand the then-ascendant Japanese business practices bet- 
ter. He wrote to dozens of Japanese companies to see if they would 
host him for a summer internship. Most never replied, but just as 
he was thinking he might be unemployed for the summer, Charles 
received a letter from a Dr. Utsumi at Canon, the camera and printer 
company. Canon was prepared to hire Charles as its first western 
intern, and soon he was winging his way to Japan. 


It sounds like a fun adventure, and it was, but it was also a huge 
shock. Charles was seconded to the production planning division 
in a distant Tokyo suburb, and assigned to a Canon men’s dormi- 
tory, three train lines and 90 minutes away. He couldn't speak or 
read Japanese. He was assigned what seemed at first an impossi- 
ble task: develop a model for how to site factories. He despaired— 
what did he know about where to put factories? It seemed like a 
specialist problem. 


But, with the help of a translating colleague, he began to inter- 
view the team about their experiences in different factory loca- 
tion decisions around the world. Patterns began to emerge in 
his findings. He learned which variables were involved, from 
local authorities’ incentives, to local taxation rates, wage levels, 
raw materials transportation cost, and so on, and eventually he 
figured out which were more or less important. Finally he built 
a logic tree that captured the variables, the direction or sign 
of impact, and the weight of the factors. He tested the model 
with data from past factory decisions and honed its accuracy 
with the senior team. In the end, this little model became the 
core tool used by the department to make complex factory sit- 
ing decisions! The secret was that it was a single-page way of 
seeing complicated trade-offs that had previously been buried in 
dense reports. It made the logic of the criteria clear, and opened 
weighting of variables up to discussion. 


It saved what might have been a disastrous internship, but more 
importantly, it convinced Charles of the decision-making power of 
relatively simple logical structures and processes in problem solv- 
ing. That is the core focus of this book. 


The Bulletproof Problem Solving Cycle 


Problem solving means different things to different people. When 
Rob asked his seven-year-old granddaughter how school was going, 
she said to him, “Papa, l'm very good at problem solving.” This of 
course was music to Rob's ears! Of course, she was really talking about 
doing math and logic problems in a school setting. Unfortunately, 
these essential problem solving building blocks are seldom taught 
as a systematic process and rarely in a way that addresses problems 
of everyday relevance and consequence. For us, problem solving 
means the process of making better decisions on the complicated 
challenges of personal life, our workplaces, and the policy sphere. 


The magic of the Bulletproof Problem Solving approach we intro- 
duce here is in following the same systematic process to solve nearly 
every type of problem, from linear ones to problems with complex 
interdependencies. It sets out a simple but rigorous approach to 
defining problems, disaggregating them into manageable pieces, 
focusing good analytic tools on the most important parts, and then 
synthesizing findings to tell a powerful story. While the process has 
a beginning and end, we encourage you to think of problem solv- 
ing as an iterative process rather than a linear one. At each stage we 
improve our understanding of the problem and use those greater 
insights to refine our early answers. 


In this chapter we outline the overall Bulletproof Problem Solving 
Process, introducing you to the seven steps that later chapters will 
address in more detail. We demonstrate the use of logic trees to 
uncover the structure of problems and focus on solution paths. We 
provide several straightforward cases to get readers started. Later 
chapters will introduce advanced techniques for more complicated 
and uncertain problems. 


The Bulletproof Problem Solving Cycle 


The bulletproof problem solving process is both a complete pro- 
cess and an iterative cycle. This cycle can be completed over any 
timeframe with the information at hand. Once you reach a prelimi- 
nary end point, you can repeat the process to draw out more insight 
for deeper understanding. 


Learn the Bulletproof Problem Solving Approach 


We often use the expression, “What's the one-day answer?” This 
means we ask our team to have a coherent summary of our best 
understanding of the problem and a solution path at any point 
in the project, not just at the end. This process of creating active 
hypotheses is at the heart of Bulletproof Problem Solving. It can 
even help you face the dreaded “elevator test.” The elevator test is 
when you, as a junior team member, find yourself in an elevator with 
the most senior person in your organization and they ask, “How 
is your project going?” We have all had this happen. You panic, 
your mind goes blank, and you stammer out a nonsensical dog's 
breakfast of an answer. The bulletproof problem solving process in 
the following pages can help you beat this situation and turn the 
elevator test into an opportunity for promotion. 


The kind of problem solving we describe can be done alone or in 
teams. If you're tackling a problem by yourself, we suggest building 
in review processes that you can use with family and colleagues to 
get the higher objectivity and other bias-fighting benefits of ateam. 


The seven steps are introduced in Exhibit 1.1. 
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EXHIBIT 1.1 The bulletproof problem solving cycle 


The Bulletproof Problem Solving Cycle 


STEP |l: DEFINE THE PROBLEM 


DEFINE When a problem's context and boundaries aren't fully described, 
We PROBLEM there is a lot of room for error. The first step in our process is 
to arrive at a problem definition that is agreed upon by those 

involved in making a decision. We test the problem definition 

> against several criteria: that it Is specific, not general, that we 


can clearly measure success, that the definition is bounded both 
in time frame and by the values of the decision maker, and that 
it involves definitive action being taken. This step may appear 
constraining, but it leads to the clarity of purpose essential for 
good problem solving. 


STEP 2: DISAGGREGATE THE ISSUES 


Once the problem is defined, it must be disaggregated (or broken 
down) into component parts or issues. We employ logic trees 
of various types to elegantly disassemble problems into parts 
for analysis, driving from alternative hypotheses of the answer. 
There is both an art and science to ‘cleaving’ problems—revealing 
oO their fault lines—that drives better solutions. This is the stage 
\ at which theoretical frameworks from economics and science 
provide useful guides to better understanding the drivers of 
your problem solution. We usually try several different cuts at 
disaggregation to see which yields the most insight. 


STEP 3: PRIORITIZE THE ISSUES PRUNE THE TREE 


The next step is to identify which branches of the logic tree have 

\ the biggest Impact on the problem, including which you can most 

H affect, and focus your initial attention on these. We employ a 

Doa 5 simple matrix of size of impact of each lever and ability to move 

S the lever as a way to prune our logic trees. Prioritizing analyses 

` helps us find the critical path to the answer efficiently, making 
7 the best use of team time and resources. 
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STEP 4: BUILD A WORKPLAN AND TIMETABLE 


Once the component parts are defined and prioritized, you then 
have to link each part to a plan for fact gathering and analysis. 
This workplan and timetable assigns team members to analytic 
tasks with specific outputs and completion dates. We'll show you 
best practices in workplanning to move quickly and accurately 
to solutions. A good workplanning process also includes team 
norms around generating a diversity of views, use of experts, 
role-playing, and flattening team hierarchies to achieve better 
answers. Good team norms and process help us avoid common 
pitfalls and biases in decision making, including confirmation bias, 
sunk costs fallacies, and anchoring bias 


STEP 5: CONDUCT CRITICAL ANALYSES 


R 


4 
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Data gathering and analysis is often the longest step in the 
process. For speed and simplicity we start with simple heuristics— 
short cuts or rules of thumb—to get an order of magnitude 
understanding of each problem component, and to assess 
priorities quickly. This helps us understand where we need to do 
more work, and especially when and where to use more complex 
analytic techniques, including game theory, regression, Monte 
Carlo simulation, and machine learning. Dont worryl Complex 
techniques are rarely needed, and when they are, new online 
analytical tools make them much more accessible than you think. 
To keep the team on the critical path we make frequent use of 
one-day answers that summarize our best understanding in the 
form of situation, observations, and initial conclusions—and team 
review sessions to pressure-test these hypotheses. 


STEP G: SYNTHESIZE FINDINGS FROM THE ANALYSIS 
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Problem solving doesn't stop at the point of reaching conclusions 
from individual analyses. Findings have to be assembled into a 
logical structure to test validity and then synthesized in a way 
that convinces others that you have a good solution. Great team 
processes are also important at this stage. 


Let's Start with Some Case Studies 


STEP 7: PREPARE A POWERFUL COMMUNICATION 


x The final step is to develop a storyline from the conclusions 

aS that links back to the problem statement and the issues that 
were defined. A powerful communication will use a governing 

thought or argument that derives from your refined situation- 

observation—conclusion logic from earlier stages. This will be 
supported with your synthesized findings and assembled into 

component arguments that may follow inductive or deductive 


logic. It will either lead with action steps, or pose a series of 
questions that motivate action, depending on audience receptivity. 
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Prepare for an Avalanche of Trees! 


We use logic or issue trees to visualize and disaggregate problems. 
We employ several types, including hypothesis trees and decision 
trees, as you will see in the cases we present throughout this book. 
We learned the power of logic trees at McKinsey and continue to 
find them essential to good problem solving. Why? Because they 
do the following: 


e Provide a clear visual representation of the problem so that 
everyone can understand the component parts. 


e Done correctly, they are holistic in the sense that everything rel- 
evant is captured in the tree. 


e Leadto clear hypotheses that can be tested with data and analysis. 


Our logic trees are sometimes simple and sometimes highly com- 
plex. But they all started on a sketchpad or a whiteboard. 


Let's Start with Some Case Studies 


To illustrate the bulletproof problem solving process, we chose 
some case studies that represent classes of problems that many of 
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our readers will face, and that exhibit the power and utility of the 
process described in detail over the next several chapters. 


1. Is Sydney airport capacity adequate for the future? 
2. Should | install solar panels on my roof now? 

3. Where should | move? 

4. Should a start-up raise its prices? 

5 


. Should | support a K-12 school education levy in my town? 


These relatively simple cases will outline each of the seven problem 
solving steps, but with a focus on the use of logic trees to help you 
represent the problem and break it into manageable parts. Later 
chapters will go into the fine points of the other steps in more detail 
and for more complicated problems. 


Case 1: Does Sydney Airport Have 
Adequate Capacity? 


When Rob was the lead partner in recruiting for the Australian 
and New Zealand practice of McKinsey, the consulting firm made 
the decision to look beyond traditional hires with MBAs to try to 
attract clever physicists, scientists, lawyers, engineers, and liberal 
arts graduates. Discussing business cases in interviews put many 
of the potential hires at a disadvantage. So his recruiting team 
came up with a non-business case that they called the Sydney 
Airport case. It is pretty simple, but it is a good way to show the 
seven-steps method. 


All of the candidates had flown into Sydney Airport and were 
aware of discussions in the newspapers about whether another 
airport was needed at that time. Sydney Airport has two of the 
10 busiest air routes in the world, so this is a real-world exam- 
ple. At the interviews the candidates were given a simple problem 
definition (step 1 problem definition): “Will Sydney Airport capac- 
ity be adequate in the future?” and asked how they would think 
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about that question. The problem statement was bounded around 
passenger airport capacity, so the candidates didn’t have to spend 
a lot of time on policy factors that might warrant a second airport, 
such as greater accessibility, safety, or environmental factors like 
noise, or even alternatives like a very fast train link between major 
cities. As we'll see later, the boundaries on problem definition are 
really important to agree on up front. 


Candidates would often ask a clarifying question or two and then 
outline their approach to addressing the issue. So what was Rob's 
team looking for? They wanted to see if the candidates used a logi- 
cal structure to help them solve the problem. It's much easier to 
show the parts of the problem in written form, so we encouraged 
candidates to use a whiteboard or pad of paper. It is usually a trial 
and error process to get the breakdown right to solve the prob- 
lem. This is step 2, problem disaggregation, and Exhibit 1.2 shows 
a simple first cut. 


SYDNEY AIRPORT 
FIRST BREAKDOWN 


Supply 
Will airport 
capacity be A 
adequate? 

Demand 


EXHIBIT 1.2 
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In this case, the simplest possible way to cleave the problem is to 
define airport capacity as supply (of landing slots) less demand. 
You could have a more complicated tree with competition from 
other ways to get to Sydney (and you might get extra credit for 
showing how those affect demand), but it probably isn’t necessary 
in this relatively simple case. 


A good candidate would dig a little deeper of course. Exhibit 1.3 
shows one way of defining airport supply capacity (number of 
runways, capacity of each runway, and utilization) and demand 
(Sydney's share of regional demand). In the short term, the number 
of runways is fixed, and so is runway capacity (defined mostly by 
aircraft type). 


SYDNEY AIRPORT 
SECOND BREAKDOWN 


Number of Runways 
x 
Supply Capacity / runway 
x 


Runway utilization 
Will airport 


capacity be 
adequate? Regional Market 


Demand 
Demand xX 


_ 


Sydney Market 
Share 


EXHIBIT 1.3 


Case 1: Does Sydney Airport Have Adequate Capacity? 


Candidates would typically explain their approach to modeling 
demand growth by making different assumptions about gross 
domestic product (GDP) growth, fuel costs, and relative loca- 
tion attractiveness of Sydney relative to other destinations (see 
Exhibit 1.4). 


SYDNEY AIRPORT 
THIRD BREAKDOWN 


Number of Runways 


x 
Supply Capacity / runway Operating hours 
x x 
Runway utilization Plomes 1 hour 
Will airport 
capacity be a 
adequate? Regional Market People / plane 
Demand 
Demand x 
Sydney Market 
Share 
EXHIBIT 1.4 


But the most productive approach to this problem is to go deeper 
into runway utilization, as it is one of the few variables that can be 
actively managed by transportation planners. Runway utilization is 
determined by hours of operation, spacing between aircraft move- 
ments, and the number of people per airplane. Hours of opera- 
tion are limited by curfew periods, weather, and maintenance. 
Thinking about how you could vary these is the heart of steps 4 
(workplanning) and 5 (analysis). 


The answers Rob most liked were the ones where candidates would 
say something along these lines: 


Runway utilization is the key so l'd be looking at operating hours, 
planes per hour, and the people per plane. You probably can’t do 
much with operating hours because there are curfew restrictions 
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between midnight and 6 a.m. because of the residents nearby. 
With planes per hour—a core variable for utilization—l’d want 
to see if they could safely reduce further the time between take- 
offs and landings. The third factor is the people per plane and 
that comes down to slot pricing favoring larger planes and policy 
about light aircraft use at peak hours (steps 6 and 7 synthesis and 
storytelling) 


Good candidates might also propose to raise prices to curtail 
demand, a tool for airport capacity management, though one 
that could result in Sydney market share loss, which city economic 
planners might not embrace. 


The branches on this kind of simple logic tree are joined together 
mathematically, so it is possible to model simple scenarios and 
show different alternatives by modifying the variables that plan- 
ners could affect. A really outstanding candidate might show the 
impact of increasing utilization by 20% on passenger numbers, or 
employing larger planes. 


What actually happened at Sydney Airport? Sydney got a third run- 
way some years later and has managed the impact of significant 
traffic growth by working on the key variables identified in the case. 
Despite the current airport authority's opposition, Sydney is to get 
a second airport in the next decade. 


Case 2: Should Rob Install Solar Panels 
on His Roof Now? 


A few years ago Rob thought it might be time to install solar pan- 
els at their house in the Australian countryside. Rob and his wife 
Paula wanted to do something to offset their carbon footprint for 
some time, but were struggling to make a decision with reducing 
(and now eliminated) subsidies available from the power company, 
declining costs of installing solar PV, and questions over the future 
level of feed-in tariffs (the price at which the electricity company 
buys from you when you generate excess power at home). Was now 
the right time? He decided to approach it in the way he had learned 
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at McKinsey and started with the hypothesis, “We should install 
solar PV now.” He hadn't reached a conclusion by framing it this 
way, nor was he setting out to confirm it without regard to the facts. 
He was using the hypothesis to bring forth the arguments to either 
disprove it or support it. 


Rob felt that the hypothesis would be supported if the following 
criteria could all be sustained: 


e If the payback on the investment was attractive, something less 
than 10 years. 


e If the decline in the cost of panels was slowing down such that 
he should not wait and make the investment later at substantially 
lower cost. Rob felt that if solar panel costs were going to con- 
tinue to decline and be significantly cheaper in three years, he'd 
consider waiting. 


e If the reduction in his CO, footprint was material, by which he 
meant 10% or more (other than air travel he is required to do and 
can offset independently). 


Rob knows that constraining the scope of the problem with clear 
boundaries makes problem solving more accurate and speedy 
(step 1). 
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This kind of problem sounds quite complex at first, a jumble of 
unfamiliar terms like feed-in tariffs and avoided carbon. A logic 
tree helped Rob see the structure of his problem in one picture, 
and helped him break up the analyses into manageable chunks. 
He started by laying out the reasons and supporting facts that he 
would need to resolve the issue. You can also think of it this way— 
for Rob to answer the question affirmatively what would he have to 
be convinced of? What are the major reasons for going ahead and 
installing solar panels? Exhibit 1.5 is a first cut of Rob’s logic tree 
(steps 2 and 4). 


FIRST CUT: SOLAR PANEL LOGIC TREE 


What do panels and 


installation cost upfront? 
Is the payoff 


attractivee 
How much will Rob save in less use 
of external power and from selling 


IF rondes Ne excess power into the grid? 


Should Rob going to keep declining. 
install solar should Rob wait until a 
panels on his later date? 
roof? 
What is Rob's total carbon 
Will solar panels mitigate footprint from his current 
Rob's carbon footprint energy user 
enough to be worth the 
hassle? How much carbon will 
Rob avoid emitting from a 
simple solar installation? 
EXHIBIT 1.5 


The first part he tackled was payback, because if the economics 
didn't work, the two other questions didn't need answering. 
Payback is pretty straightforward: the cost of the installed solar 
panels and inverter, divided by the annual electrical cost sav- 
ings. The denominator in this analysis includes both estimating 
net savings from the installation from avoided electricity charges 
because he was using his own power, plus income from supplying 
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electricity to the grid via feed-in tariffs. Most of this analysis can 
be done by online calculators that solar installers offer, once 
you know the size of the system, roof orientation, solar electric 
potential, and the efficiency in power generation. Rob simplified 
the analysis by leaving out battery storage options that add to 
cost but provide the opportunity to replace peak power charges. 
With an annual cost savings of around $1,500 and investment 
costs of just over $6,000, payback was attractive at about four 
years (step 5). 


The next question was whether he should make the investment 
now, or wait, hoping for lower solar panel costs later. Rob was aware 
that the cost of a watt of PV had fallen almost 30% from 2012 to 
2016, and almost 90% from the early days of solar PV. He wasn’t 
sure whether this would continue in the future. With some simple 
Internet research, Rob learned that declining costs of equipment 
was still uncertain, but the cost per watt was unlikely to fall by more 
than 30% for at least the next three years. There is also uncertainty 
about future feed-in tariffs that have been set to encourage sales 
of solar PV. This has to be considered against rising retail prices for 
electricity customers. 


At $1,500 per year, the cost savings lost by waiting would be $4,500 
over three years, so the up-front cost of the solar PV installation 
would have to fall by 75% to make waiting worthwhile. Rob could 
have used a net present value analysis where the time value of 
money is considered rather than a simple payback. But in this case 
the simple method is fine: He felt comfortable with the four-year 
payback providing an implied rate of return of 25%. It was worth 
doing now. 


Finally, he wanted to estimate how much of his CO, footprint he 
would reduce by going ahead. This depends on two things—one 
is what fuel source he is displacing (coal or gas in this case), and 
the second is the kilowatt hours (kWh) he is generating com- 
pared to his electricity use, which he knew from the first step. Rob 
simplified the analysis by looking at the carbon footprint of the 
average Australian citizen, and found that the avoided carbon 
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from his little solar project could reduce his footprint by more than 
20%. Since the payback as an investment is very solid in this case, 
Rob really could have pruned off this branch of the tree (step 3) 
and saved some time—but he and Paula had multiple objectives 
with this investment. 


Whenever you do this kind of analysis, it is worth asking what could 
go wrong, what are the risks around each part of the thinking? 


PROBLEM DISAGGREGATION: 


Cap Ex 
$5,507 
Investment 5 
$6,307 Inverter replacement ize of system 
PRETTY ATTRACTIVE! (once during lifetime) 
. - $1300 Solar electrical 
potential 
kWh 2135 
k generated Efficiency in power 
* Is the payback 9073 generation and capture 
attractive? 
4 years Roof orientation/ 
F slope North facing 
Avoided Rae ae Slope: 40° 
$1,058 $023 
% of Demand side 
Netcare AT response (DSR) 
$1557 used Storage 
50% Smart 
SHOULD | appliances 
PURCHASE AND Savings from 
feed-in tariffs 
INSTALL SOLAR $499 
PV NOW? kWh generated 9073 
Export tariff $011 
Will the price ai 


continue to decline? 
Should we wait? 


drop by 75% to make it 
worth waiting. 


Will this be a 
signifant contribution 
to mitigating climate 
change? How many kg 
of CO, emissions will 
be avoided? 


` * + +7803 ko CO, I kWh 


As a percentage of the carbon 
footrpint of the average 
Australian, this is about 29% +., 


~ PRETTY SIGNIFICANT! 


Exhibit 1.6 


p °°. Oe Panel prices would have to 


% exported 50% 


Emission factor of 
NSW grid electricity 
(kgCO2 IkWh) 


0.86 kg C02 | kWh 


kWh generated 
9037 kWh 


Assumptions: House type: detached bungalow, Roof orientation: north, Slope: 
40°, Suitable roof area: 40m2, Installation size: 5kWh, Shading: none, Numbers 
for calculations are from 2017, Australia. 


Case 3: Where Should | Move? 


In this case there is a chance that the power company would reduce 
the subsidies for installing solar PV. This can be mitigated by acting 
quickly. The power company could also reduce the feed-in tariff rate 
at which it purchased any excess power produced by Rob—and in 
fact they did that later. But with a four-year payback the exposure is 
reasonably limited. 


The result of Rob’s analyses is shown in the more complicated tree 
shown in Exhibit 1.6. 


With only a bit of online research, Rob was able to crack a rela- 
tively complicated problem. Rob should install solar panels now. 
The payback is attractive, and likely cost declines to install later are 
not enough to offset the savings he could earn now. As a bonus, 
Rob and Paula were able to reduce their carbon footprint by nearly 
30% (steps 6 and 7). 


The core of this good result was asking the right questions and 
disaggregating the problem into straightforward chunks. 


Case 3: Where Should | Move? 


In the early 2000s Charles was living in Los Angeles. Having recently 
sold the company he cofounded, his family wanted to move to a 
small-town environment where there would be more opportunities 
for recreation and really good schools. They liked the ski towns 
they had visited, and they had always enjoyed college towns. But 
how to choose? There are so many variables involved, and it is 
easy to get it wrong with only impressions from brief visits. Then 
Charles remembered the factory siting problem he worked on 
back at Canon in Japan and set up the decision-making effort in a 
similar way. 


The whole family got involved in the problem solving brainstorm- 
ing, kids included. They started by listing out what mattered to 
each of them, so their personal factors defined what it meant 
to be a good place to live. The family agreed on a weighting 
that favored the school system first, then the natural environ- 
ment and recreation, and finally characteristics that made for 
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FAMILY BRAINSTORMING SESSION: WHERE TO LIVE? 


ELEMENTS OF A GOOD LIFE WHAT DOES THAT LOOK LIKE? 


Really good schools for the kids | Great teachers 
© Small class sizes 
Good taxpayer support for education 
School choices: public, charter, private 
Graduates get into good colleges 


A clean environment and lots to do ; Water and air quality high 
outside : A four season climate 
» Lots of sunny days, but sufficient rain 
Rivers to fish in 
Great hikes nearby 
Skiing and mountain biking 


A cool, friendly town A walkable town center 
| Arts, theatre, libraries 
Not too much traffic 
Fun coffee shops and good restaurants 
Do any friends live there? 
ls it a university towne 
ls crime a probleme 


Can you earn a living? : Cool, small companies 
: Diverse local economy 
Not too far to West Coast work for Charles 


EXHIBIT 1.7 


a cool, in-town experience. Charles then added the elements 
of the ability to earn an income! These were agreed after lively 
debate with everyone involved (step 1). They planned to use the 
list to develop a set of towns to visit during family vacations (see 
Exhibit 1.7). 


Charles began the analysis by breaking down the problem into 
the major elements the family said they valued, then identifying 
subelements, and finally measurable indicators or variables that 
captured each subfeature, such as sunny days or a comfort index 
(defined by temperature and humidity) for climate variables (step 4). 
It was a little work, but he discovered that most of the data was 
available online. With the family’s input, he put a relative weight 
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next to each variable, to reflect the importance of each element to 
their final decision. 


He developed a tree of around 20 variables and gathered data for 
about a dozen towns (step 2). The tree he developed is shown in 
Exhibit 1.8, with the weightings shown in red. 


WHERE TO LIVE: PART | 


WHERE 
SHOULD WE 
LIVE? 


EXHIBIT 1.8 


Really good 
schools 35% 


A nice 
environment 


25% 


A cool, 
friendly 
town 


25% 


Not too 
far from 
everywhere 


15% 


Achievement index 20% 


Good private schools 15% 


A pleasant climate 9% 
Natural beauty 8% 


Recreation close by 8% 


Quality of town center 4% 
University town 5% 
Nice size: not too big 4% 
Arts center 4% 


Interesting small companies 4% 


Presence of friends/family U% 


Average rainfall 2% 
# of sunny days 3% 
Snow fall 1% 
Comfort Index 3% 


Watershed quality 15% 
Air quality 15% 
Subjective appeal 5% 


Fishing 3% 
Hiking 3% 
Kayaking 1% 


Mountain biking 1% 


Distance to west coast 15% 
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Once Charles gathered the data on a set of small college cities and 
mountain towns, it became clear that some of the variables were 
repetitive and others didn't really help distinguish between loca- 
tions. He pruned his tree to make the analysis simpler and faster. 
It meant some locations didn’t need to be visited. It also revealed 
that some factors he felt were important around airports and hubs 
could be encompassed in a single measure of commute time from 
each town to the West Coast, where most of Charles's work with 
young companies was located. A variable included early in the 
analysis was community safety or crime, but that turned out not to 
be a differentiating factor among the preferred communities, so it 
was also pruned (step 3). 


He converted all the data for each factor to a common scale 
from 1 to 100, and then applied the weightings. There are vari- 
ous approaches to what is called normalizing data, but they are 
straightforward and can be found online. As you can see, some of 
the variables have a positive slope (for example, more days of sun- 
shine is good), and some are negative (for example, longer com- 
mute times). So a 100 for Amherst, Massachusetts, on travel time 
is a negative weight in this case. If we were to get fancy, you could 
have more complex line shapes to the weighting for variables like 
rainfall, where you want some rain, but not too much! Exhibit 1.9 
shows Charles's analysis (step 5). 


In this case, the family was able to reach a conclusion to choose 
Ketchum, Idaho (steps 6 and 7). They all agreed on the choice 
because they had agreed on the factors that made for a good loca- 
tion and how to weigh or trade them off. There was a big trade-off 
in moving to Ketchum, Idaho that Charles was prepared to accept— 
the commute time to the West Coast for business was longer than 
other locations. 


We sometimes hear a criticism with systematic problem solv- 
ing like this example that you went through a phony process 
to prove what you had in mind from the beginning. In this case 
it isn’t phony at all: Ketchum wasn’t even on the list of the ini- 
tial towns under consideration and was only added after a trip 
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AND THE BEST TOWN IS... 


Ce A e wv S S F S 
oe oe = oe a Q s À * 
NORMALIZED TABLE W gag gs a ge QE GM A 
Avg rainfall per annum 2h 7) 24 0 58 82 29 «100 $a 
# of sunny days 3% 100 91 So) 74 35 79 $a 
Snowfall (inches) 1% 0 4 y 36 36 100 #7 
Watershed quality (best=100) Z) 7 ei 100 35 9% E 
Air quality (best=100) 2% 33 36 33 38 %3 %2 E 
Comfort index (best=100) 3% 30 46 A 100 O 86 80 


Air time to West Coast (worst=100) be 0 V 33 WV 33 100 50 Hie 
Availability of private schools (index) 15% 1 


Achievement index 20% 46 0 38 65 97 100 78 Mime 
Presence of Friends/family u% 100 0 0 0 0 0 0 00 
University/college town 5% 0 0 100 100 100 100 0 50 
Arts center Ute sl sy XC) 100i (00) oo 15 
Quality of town center E 33 0 67 67 50 50 100 
Physical beauty of area Sa 2 e 30 Se 100 100 
Interesting small companies 4% 60 O 40 60 100 80 O 20 
Size US O S05 A e 
Recreation 8 50 40 60 50 70 20 60 En 
100% 
TOTAL WEIGHTED INDEXED SCORE Wg 20 34 51 70 70 52 ie 
EXHIBIT 1.9 


to visit friends. But it is worth pointing out some risks in this 
analysis too. Boulder and Amherst are really close in total score 
to Ketchum, and quite small changes in how Charles ranked 
factors such as recreation or quality of town center—which are 
subjective—could have a big impact on the numerical conclu- 
sion. In this case the family could mitigate the risk by visiting 
each town and testing their feelings about the quantitative 
variables firsthand. 
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The where-to-live case illustrates how you can start with a simple list 
of issues or elements that are related to your problem statement, 
disaggregate the elements further into indicator variables, then 
finally add concrete measures and weights. The rest is straightfor- 
ward arithmetic based on a considered ranking of features. This type 
of tree and analysis approach has applicability to many choice prob- 
lems. Charles and Rob have used it to assess what apartment to buy, 
what employer to join—and, of course, where to put your factory. 


Case 4: Making Pricing Decisions 
in a Start-up Company 


In the past few years, one of Charles’s friends started a company 
that makes an accessory for pick-up trucks that has a unique and 
clever design. The company, which we'll call Truckgear, sells around 
10,000 units a year, a number that is growing quickly. It is at break- 
even on a cash basis (cash basis means not taking into account the 
accounting charge for depreciating assets). Charles invested in the 
company and helps devise its strategy. 


Start-up companies face big and complex problems early on in the 
process and, compared to larger companies, they have limited cash 
resources and team members to address them. Truckgear had to 
make decisions on whether it should own its own manufacturing 
plant, which market segments to compete in (there are new and 
used truck segments and several sales channels to each), whether it 
should have its own sales force, how much to spend on marketing, 
and most fundamentally, how fast to grow given limited cash? No 
wonder start-up teams hardly sleep! 


Recently the company had a big decision to make: Should it raise 
its prices (step 1)? It had held its initial pricing of around $550 for 
three years. Materials and manufacturing costs had increased as 
the product features were improved, crimping its margins and low- 
ering the cash generated per unit. Obviously in young companies 
cash is even more critical than in established ones, as the sources 
of external financing are significantly fewer. The dilemma Truckgear 
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faced was this—if the marketplace reacts negatively to the price 
increase, Truckgear growth would slow and perhaps even drop in 
unit sales. 


There is no perfect answer to this kind of question, but we employed 
a particular kind of logic structure to assess it, a profit lever tree 
(step 2). We wanted to hone in on the key factors around the deci- 
sion, and this kind of tree is mathematically complete, so we could 
use it to model different assumptions. 


Exhibit 1.10 is a simple version of this kind of tree. 


TRUCKGEAR PROFIT LEVER TREE 


U. Without slowing 3. Can we raise 
sales growth or price per unite 
dropping units? 


5. Units sold are affected by: 
» Competitor prices 


2. This pushes down 

variable margins 

» Customer price sensitivity 

» Dealer channel margins 
required 

» Marketing and sales efforts 


New truck dealer 


Used truck 


Average price < 
Be accessory dealers 


_ per unit 
Direct online 
i sales 
otal =e “4 
Revenue y 
ite Gell — = Variable Unit Margin 
TOTAL 
a a Variable X 
PROFIT mee 5 
\ \ Cost of materials 
\ Variable / A 
cost per = - Manufacturing cost 
\ Total + unit a 
Costs Jie 
Staff 
\ Fixed costs + 
aa S ae 1. Pressure here 
Other 


EXHIBIT 1.10 
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You can see how the tree makes Truckgear's problem visual: Pres- 
sure on costs pushes down variable margin per unit—can the com- 
pany increase unit prices without slowing sales growth or even 
dropping volume? 


Exhibit 1.11 displays the numbers for Truckgear. 


TRUCKGEAR PRICING DECISION 


New truck dealer 


Used truck 


Average price 
accessory dealers 


per unit 
$550 Direct online 
Total sales 
Revenue = a 
$5.5m 
2 = Variable Unit Margin 
Units sold ° 41% 
TOTAL 10.000 
CASH = = Variable 
PROFIT TEE aN * 
$3.25m Cost of materials 
$350,000 $120 
Variable 
Total ate per = Manufacturing cost 
ota uni 
oe + $305 + es 
$5.15m ys 
Staff 
Fixed costs + 
and = Rent 
overhead + 
$1.9m Other 


EXHIBIT 1.11 


If the company could hold its current unit sales, a price increase 
of 7% would yield a $385,000 improvement in cash profitability, a 
substantial increase that could help fund additional marketing and 
sales programs. But you can also see that it would only take a drop 
in unit sales of 650 to neutralize the benefit of the price increase 
(step 5). What to do? 
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Whether a price increase leads to a loss in total cash profit (or, less 
seriously, slowing of growth) depends on competitor pricing, cus- 
tomer price sensitivity (which economists call price elasticity), 
whether the third-party dealer sales channels will absorb some of 
the price increase by accepting lower margins, as well as marketing 
and sales efforts. The company initiated a large phone survey of its 
recent customers and determined that: 


e The largest volume customer segments were not sensitive to a 
modest cost-based increase. 


e The competitor products were roughly comparable in price and 
quite different in functional characteristics. 


e Dealers were not willing to reduce their margins to accommo- 
date the price increase. 


The company also evaluated whether it could achieve the same 
result by reducing its fixed overheads or taking manufacturing 
in-house. With few costs other than a lean staff and rent, the first 
was not an option. With limited current cash resources, investing in 
its own extremely expensive manufacturing presses and assembly 
also didn’t make sense (step 3). On balance a small price increase 
to restore unit margins was worth the risk (step 6 and 7). 


This kind of financial tree is particularly useful for solving problems 
that involve monetary trade-offs of alternative strategies. You can 
use it to track almost any kind of business problem. We'll show a 
number of more sophisticated versions in later chapters. 


Case 5: Should Charles Support the Local 
School Levy? 


In Charles's former hometown in Idaho, public education funding 
is supported principally by real estate taxes, which are levied as a 
percentage of property value each year, and by state sales taxes. 
When a local school board has bigger strategic investments to fund, 
it seeks approval through a vote by taxpayers for an additional levy 
to pay down a bond secured for that purpose. In the late 2000s 
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the Blaine County School Board proposed a bond of more than 
$50 million to support a large set of investments across the county. 
With a population of only 20,000 people, depending on the size of 
your property, the levy could cost a homeowner thousands of addi- 
tional dollars per year. 


As taxpayers and citizens, people face this kind of decision all the 
time—a spending levy, a one-time referendum, or whether to sup- 
port a candidate pitching a new state or national policy. The problems 
seem complex and are often swayed by partisan debate rather than 
informed by straightforward problem solving that any of us can do. 


Charles was aware of press stories that K-12 education in the United 
States is lagging behind its peers globally, and so was in principle 
supportive of additional taxes on his house if it really improved local 
education. But he wanted to know whether voting for the school 
bond would really have an impact on closing the gap locally (Idaho 
ranked in the bottom half of US states in school-testing results). 


Charles is no expert on education policy. He started by asking a 
simple question: What is the key problem with K-12 education in 
the US versus its peers, and does the proposed school levy address 
these issues (step one)? He knew the United States spends a lot per 
student in most jurisdictions, among the highest in the world, and 
he knew overall results were middle of the pack compared to peer 
countries. So he asked, is the problem: 


e Per-student funding? 
e IQ or demographics? 
e Teachers and schools? 


His research showed that the answer is mostly with teachers and 
schools. There was nothing in the data to suggest either that fund- 
ing levels per student (higher in the United States than most coun- 
tries) or student IQ levels (comparable to other countries) were the 
reason for poor student outcomes in the United States (step 3). 
Study after study shows that student outcomes on international 
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test scores vary widely and are best explained by teacher char- 
acteristics and the school environment. Charles then proceeded 
to research which factors about teachers and schools had most 
impact on student outcomes and found four, ranked in approximate 
importance (step 4): 


e Teacher numbers and classroom size 
e Teacher quality (education, experience, training) and compensation 
e School environment and facilities 


e Technology 


Next, he overlaid the extent to which the proposed bond would 
provide funding against these factors. It turned out that the fund- 
ing was mainly directed at clean energy and school facilities, 
low-impact factors. There was little allocation of funding to the 
higher ranked factors, especially related to teacher recruitment, 
pay, and training (step 5). Charles decided not to support the bond 
on these grounds. Exhibit 1.12 shows what the analysis looks like in 
a decision tree format. 


This analysis illustrates bulletproof problem solving for a societal- 
level policy problem: It takes a real issue, frames a question fol- 
lowed by a series of sub-questions, and guides research and 
analysis. Charles collected facts about education performance and 
the details of the school bond planned spending allocation. It took 
him only a few hours of framing and online research and helped 
move him from an emotional decision (“| support education and am 


concerned about local school performance,”) to a reasoned one ("1 
can't support this levy, given how it is allocated.) (steps 6 and 7). 


On the third attempted school levy a few years later, the measure 
contained funding for early childhood education and more teacher 
training—and the vote passed. 


The next chapters will examine each of the seven steps to bullet- 
proof problem solving in more detail, introducing more complex 
problems, and more sophisticated approaches to solving them. 
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K-12 EDUCATION LEVY IN IDAHO 


K-12 education in the United States is lagging behind its peers (globally). Will voting 
in favor of the local school bond have an impact on closing the gap (locally)? 


WHAT IS THE KEY PROBLEM WITH K-12 EDUCATION IN THE UNITED STATES? 


Is it per student funding? | 


Yes 


Is it IQ scores or demographics? 2 


N 
: Yes 


Is it teachers and schools? > 


Yes 
No 
So, what factors 
can we improve? 
Teacher quantity + Teacher Environment Technology 
classroom size, quality, + facilities . + capital 
What is the estimated impact of : 
this factor on closing the gap? low-medium high : low medium 
Does the school levy provide 


funding for this factor? " 


1 OECD PISA scores 
710 Research 
3 How the World's Best Performing School Systems Come out on top, McKinsey and Co, Sept 2007 


" School board approves October levy election, Terry Smith, 2009 


EXHIBIT 1.12 


Chapter 1 Takeaways 


e Good problem solving is a process, not a quick mental calcula- 
tion or a logical deduction. It applies to even highly complex 
problems, like where to put factories around the world, as Charles 
experienced as an intern at Canon. 


e The problem solving process we outline in the book is a cycle of 
seven steps that you work through; each step is important and 
lots of mistakes result from skipping steps. 


Problems to Try on Your Own 


e The most important step is to disaggregate the problem in a 
logical way into component parts, so you can isolate the most 
important analyses; logic trees are our prime work tool—they 
make it easy to see the structure of the problem. 


e Prioritizing analyses is essential so that you avoid working on parts 
of the problem that don't contribute much to your answer—we 
call this being on the critical path. 


e A well-defined work plan is needed to allocate analysis to 
team members and time frames for completion (in this chapter 
not much work planning was required because the problems 
were simple). 

e How you go about analysis, using simple tools or sophisticated 
ones, is important to problem solving success; we always start 
with simple estimates and heuristics or rules of thumb. 


e Problem solving isn’t over until you can synthesize the results of 
your analysis and tell a story that convinces someone to act. 


e We use different types of logic trees for different problems; in 
this chapter we have shown mathematically complete deduc- 
tive logic trees for business problems, weighted factor analysis 
for making decisions, and decision trees for walking through 
complex choices. 


Problems to Try on Your Own 


1. Individual: Show the logic tree for whether you should make a job 
change—try a decision tree or a factor-weight tree (like where to 
live, but with job characteristics that are important to you); or try 
it for a new house or apartment, with weighted characteristics 
you would look for, and compare them against listings near you. 


2. Business: Lay out the profit tree for your business or social enter- 
prise to the third breakdown. 


3. Societal: Draw a logic tree for Britain's decision whether or not to 
leave the European Union (the Brexit decision). 


29 


Chapter Two 


Define the Problem 


DEFINE 


A TWE PROBLEM 


5 


Getting a crystal-clear definition of the problem you are solving 
is the critical starting point to bulletproof problem solving. And it 
should be relatively straightforward. But a surprising number of fail- 
ures in problem solving originate in poor problem definition. Teams 
and individuals surge off into data gathering or expert interviews 
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without being very clear about the boundaries of the problem, the 
criteria for success, the time frame, or level of accuracy required. 


This is bound to lead to disappointment. A scattergun approach 
to data gathering and initial analysis always leads to far more effort 
than is necessary. And the answers often miss the mark with decision 
makers whose values and boundaries need to be taken explicitly 
into account in problem solving planning. 


Charles saw this kind of mistake first-hand in the 1990s when he was 
running Citysearch, an Internet city guide company that sometimes 
collaborated with local newspapers and sometimes competed 
with them. Newspaper companies were the kings of local advertis- 
ing then and had been for 100 years, despite the earlier advent of 
first radio and then television. Surely, the little Internet company 
entrants would be no different? 


Charles met with many senior leaders in the newspaper trade 
during those years and witnessed a kind of arrogance rooted in 
inaccurate problem diagnosis. These managers believed that the 
problem statement around new competition was: “Do the new 
Internet entrants have better content (news and other writing) that 
would win over readers?” They concluded that their newspapers 
had editorial content that the entrants couldn't match and there- 
fore they didn't need to worry too much. When Charles was invited 
to give a keynote presentation to the 1997 Newspaper Association 
of America annual meeting and suggested that the competitive 
attack would instead come from behind, from erosion of near 
monopoly positions in classified advertising in segments like autos, 
real estate, employment, and personals, the room was entirely 
silent. This upstart was telling them about their market in a way that 
made no sense to their worldview. At the end no one clapped—it 
was awkward! 


Over the next few years it played out as predicted. New verti- 
cal players in online classified advertising sprang up and slowly 
cannibalized the long-held newspaper dominance in these catego- 
ries. Consumers still wanted newspaper editorial content, but they 
began to find their jobs, cars, homes, and dates online, stripping 
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away the key source of revenue for old-line media companies. 
Failure or consolidation of many great newspaper names followed. 
This turns out to be common in cases of disruptive technology or 
business model innovation. The existing players literally cannot see 
the threat from new entrants because their mindset, their problem 
boundary definition, makes them blind to it.’ The takeaway point 
here is clear: Getting problem definition right, including bounda- 
ries, is essential to good problem solving and can be an essential 
competitive advantage. 


Good problem statements have a number of characteristics. 
They are: 


e Outcomes focused: A clear statement of the problem to be 
solved, expressed in outcomes, not activities or intermedi- 
ate outputs. 


e Specific and measurable wherever possible. 
e Clearly time-bound. 


e Designed to explicitly address decision-maker values and bound- 
aries, including the accuracy needed and the scale of aspirations. 


e Structured to allow sufficient scope for creativity and unex- 
pected results—too narrowly scoped problems can artificially 
constrain solutions. 


e Solved at the highest level possible, meaning for the organiza- 
tion as a whole, not just optimized for a part or a partial solution. 


Teams we worked with sometimes used a mnemonic—SMART—to 
remember these characteristics: whether the problem statement 
was specific, measurable, action oriented, relevant, and timely. 
SMART covers most but not all of these factors—make sure to have 
an outcomes focus and work at the highest level. 


Usually you can’t fit all those characteristics into a single statement, 
so we like to use a problem statement worksheet that captures the 
whole context, as in Exhibit 2.1. 


'See, for example: Clayton M. Christensen, The Innovators Dilemma: When New 
Technologies Cause Great Firms to Fail (Harvard Business School Press, 1997). 
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PROBLEM DEFINITION WORKSHEET: 


Problem statement: What are we truing to solve? 


| Decision maker(s) | Criteria/measures for successful effort | 
What audience are you addressing? How would the decision maker judge a 
Who needs to decidelact? successful problem solving effort? 

| Key forces acting on decision makers | Time frame for resolution 

\ 4 L 
What are their concerns and issues around How quickly is the answer needed? 


the decision? 
How will you address conflicting agendas? 


| Boundaries/constraints | Accuracy necessary 
l 4 L - 


What is off-limits or not under What level of accuracy Is needed? 
consideration? 


EXHIBIT 2.1 


How a Problem Statement Is Refined: 
The Case of Pacific Salmon 


Let's illustrate how this works with a specific example. Charles was 
asked by a newly formed charitable foundation, the Gordon & Betty 
Moore Foundation, to look into the question of how to preserve 
wild Pacific salmon, a hugely important species in the north Pacific 
rainforest ecosystem. The iconic salmon species in the Atlantic 
Ocean had seen huge declines from mismanagement that caused 
large-scale ecosystem harm and community economic losses, and 
the foundation's founder wanted to ensure that this didn’t occur 
in the Pacific. As you can see, this is a much larger and more com- 
plicated problem than our more narrowly bounded problems of 
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Chapter 1. But you'll see the same principles apply in solving harder 
problems—you just need to put a bit more work into each of the 
seven steps. 


Here is what the team knew: Pacific salmon are a critical element 
in northern rainforest ecosystems, sometimes called an apex spe- 
cies for the role they play in these systems. Nearly half a billion 
Pacific salmon are a substantial biomass in their own right, and 
have massive impact on both freshwater and marine ecosystems. 
They provide nutrition for many animal species, jobs for com- 
mercial fishers, food and cultural meaning for indigenous people, 
recreation for sport anglers, a huge boost to local economies, and 
are a conduit for marine nutrients to help forests thrive. It is hard to 
overestimate their importance in the environment, economy, cul- 
ture, and food webs of this huge region. And they are in trouble, 
at least in many parts of their historic range, pressured by human 
development in land use and fisheries management approaches 
that affect both habitat and salmon numbers. 


The team wanted to help reverse these declines, but frankly weren't 
sure where to start. The foundation was new, but it was already 
committed to a long-term model of philanthropy, whereby it would 
only focus on a few initiatives and would fund them for substantial 
periods. The founder is a world-famous engineer, and his personal 
values were core in developing the foundation’s four filters for 
the work it agreed to take on, one of which was to only work on 
projects with measurable outcomes. The other three filters were 
importance (working on things that really mattered); only work- 
ing on initiatives where the foundation could make a difference 
(working on things where the foundation’s unique contribution 
mattered); and, working on initiatives that over time would con- 
tribute to a portfolio effect. This means that the grants it made 
would invest in capabilities and outcomes that would build off and 
support each other. Based on this, an initial problem worksheet 
looked something like Exhibit 2.2. 
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PACIFIC SALMON PROBLEM WORKSHEET 


Problem statement: To substantially increase the number of Wild Pacific Salmon 


t 


| Decision maker(s) | Criterialmeasures for successful effort | 
L { \ 
»Head of Foundation Environment Program »Four foundation filters 
» Foundation board of trustees 1. Importance 
» Other stakeholders: environmental organizations, 2. The organization could make a 
other grant makers, government agencies, difference and have enduring impact 
fisheries, and indigenous groups 3 Outcomes can be measured 


U Contributes to a portfolio effect 
» Could team demonstrate credibility of foundation 


initiative approach? 


{ 
| Key forces acting on decision makers | Time frame for resolution 
»First initiative done by new foundation »Up to 15 years but had to prove we could design a 
» Wanted to establish a good reputation program and get traction 
» Wanted to prove a new model of philanthropy » Annual reviews to test 
i 
| Boundaries/constraints | Accuracy necessary | 
»Foundation did not favor direct political policy »Need to demonstrate that the initiative could 
work or hard-edged advocacy (science over generate ecosystem scale outcomes over a single 
politics, pragmatism over movements) generation 


»Willing to spend $10-15m per year 
» Outcomes, not activities 


EXHIBIT 2.2 


The team found this sharply defined problem context extremely 
valuable. They knew they had up to 15 years to work on the prob- 
lem and very substantial financial resources to invest if progress 
remained good. But the team had some real constraints it had to 
work within. It had to demonstrate that it was making traction on 
the problem quickly in frequent top-level reviews, and it had to 
develop strategies that would yield measurable ecosystem-level 
outcome improvements over time. It had to build portfolio effects 
of capabilities and connections that would benefit other founda- 
tion environment work. And, as the first team, it was important to 
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show to the broader grant-making and conservation communities 
that the focused portfolio approach to environmental grant making 
would really work. These are high bars, but it is really useful to know 
what the threshold for success is in advance. 


It is also critical to know what is off limits and to understand the 
implications of the boundaries to the solution set. In the salmon 
case, we knew that grassroots advocacy campaigns could not be 
a primary strategy, with a similar conclusion for large-scale direct 
policy efforts, given the foundation's lower level of comfort with 
these approaches. 


Let's turn to the problem statement itself. One of the reasons the 
foundation selected saving Pacific salmon as one of its first granting 
programs was the apparent fit with the foundation's filter of measur- 
ability, a deeply held core value. After all, you can count fish, right? 
You can see this coming: It turns out it is a little harder than it looks. 


There are five different Pacific salmon species spread around 
the Pacific Rim from California through British Columbia, Alaska, 
Russia's Kamchatka and Sakhalin regions, all the way to Japan. 
Some species are doing well in some places, many not so well. And 
overall numbers go up and down each year due to a number of 
factors. One of the biggest of these are ocean conditions that lead 
to more or less of the smaller creatures that salmon eat; these fol- 
low a rough temporal pattern related to ocean temperatures called 
the Pacific Decadal Oscillation. While that sounds like a fancy bit of 
climatology, in fact it meant that it was very difficult to separate and 
account for the causes of interannual variation in salmon numbers. 
In short, salmon aren't easy to count, and it isn’t easy to determine 
in strict numerical terms what the positive or negative impact is of a 
new policy or fisheries management program, or a new bit of pro- 
tected habitat. For the team, that meant measurement would have 
to be over longer periods of time, making estimates of the impact 
of uncontrollable factors like ocean productivity. 


As the team understanding of the issues grew via early analysis 
and work with potential grantees, the problem statement evolved 
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and improved. Most importantly, the team added more emphasis 
on maintaining salmon stock diversity and full functioning of all the 
elements of the ecosystem. Stock diversity means all the different 
ways smaller populations of the same kind of salmon use the rivers 
and oceans, eating different food and migrating at different times 
and on different routes. Diversity in the structure of the different sub- 
stocks helps the overall population weather environmental shocks 
better, and so contributes to long-run abundance. Exhibit 2.3 gives 


SALMON PROBLEM STATEMENT EVOLUTION 


Statement 


To substantially and 
sustainably increase the 


Positives 


» Simple 
» Appears measurable 


Critique 


» Measurability is 
challenging 


$5 
Z number of wild Pacific »lgnores importance of 
+> salmon species diversity 
N ‘ 
ss » Doesn't link to full 
(ite 
ecosystem health 
» Specific enoughe 
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a flavor of that evolution and an assessment of the quality of these 
problem statements. As you can see, one of the ways of address- 
ing the challenges of counting salmon is to introduce the notion 
of sufficient populations to make use of available ocean-carrying 
capacity, a measure driven by food availability and habitat. This sets 
a bar for healthy salmon numbers as the abundance and diversity 
of subspecies to utilize all of the available ocean capacity, and is a 
much more useful concept than plain numbers in any one year. 


Each of these problem statements reflects the foundation salmon 
team’s greater and greater understanding of the problem as it 
iterated the process with salmon management agencies, tribal 
groups, commercial fishing organizations, sport angler groups, 
salmon scientists and the conservation community. Constant itera- 
tion allows the team to hone its understanding and therefore to 
sharpen its strategies to achieve the desired outcome—at the 
same time keeping all the stakeholders onside as the process runs 
through time. 


This evolution also points to a discussion within the foundation 
on the right level of aspiration to aim for. A narrower regional 
program focused primarily on salmon management might have 
been quicker to gain progress compared to a full north Pacific 
ecosystem level program, but also would have had less impact on 
the overall environment and human and ecosystem well-being. 
A narrower framing might also have made it more difficult to gen- 
erate fundamental shifts in public attitudes toward conservation, 
and more challenging to attract funding from other foundations. 
On the other hand, any of these frames still bears the risk of being 
overwhelmed by bigger global environmental threats, such as 
climate change. 


Defining the Highest-Level Problem: Steel Company 
Investments 


The point of this is to find the right scale and scope in problem 
definition for your organization. When we worked for McKinsey, 
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we often saw problems that benefited from redefinition to a 
higher level. For example, in one instance Rob's team was asked 
to review a $6 billion capital equipment investment plan for a large 
steel company. It looked straightforward: Evaluate the costs and 
benefits of each investment over time. But when they dug into the 
data, they discovered that the larger issue was one of an inabil- 
ity to generate cash because of an uncompetitive cost position, 
a much more pressing issue than the planned investments in new 
plant and machinery. The team’s findings convinced the client to 
first address costs and attack the lack of cash generation by the 
business. They needed to lower overhead costs substantially and 
select only a fraction of the projects slated for investment to be 
successful. With these changes, the business was able to generate 
cash to survive. 


When possible, it is advantageous to allow flexibility in the 
scope or width of your problem solving project. This gives rooms 
for the kind of discovery described in Rob's Australian steel cli- 
ent. But just as important, it also provides room for creative and 
novel solutions. Narrowly scoped projects make for fast prob- 
lem solving, but they often lead to keeping the blinders on by 
employing conventional conceptions of the problem space. 
Breakthrough ideas are unlikely to emerge with old models and 
framing. People often wonder at the idea that newly minted 
MBAs employed by consulting firms can add value to organiza- 
tions with veteran management teams. The answer lies in the 
fresh perspectives that new eyes and new analytic approaches 
can bring to old problems. 


One of the other important principles in problem definition is to 
solve the problem at the highest level possible. The insight here 
is that the most granular and local solutions are often not optimal 
for the larger organization. In business, you often find this issue in 
doing capital allocation and investment decisions at the business 
unit level. You find what makes sense for a single unit is not what 
makes sense for the company overall. Wherever you can, target 
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your problem solving efforts at the highest level at which you can 
work, rather than solving for the interests only of smaller units. In 
Rob's steel example the client wanted to solve for the right capi- 
tal budget, but the higher-level problem definition was their ability 
to generate free cash flow to convince shareholders to reinvest in 
the business. 


Similarly, Charles’s salmon team learned that smaller regional 
approaches didn’t work very well because salmon migrate through 
other regions and congregate in common far ocean feeding 
grounds. Solutions to reduce commercial fishing or choose more 
selective gear types in one area could be undone by increased fish- 
ing pressure in others. The north Pacific scale was the right scale to 
work this problem. 


Widening the Aperture on the Problem: The Avahan HIV 
Project in India 


Good problem definition or framing can have a huge positive 
impact as we have shown in our examples. The success of the 
Avahan HIV program in India was only achieved after a substantial 
reframing and rescoping. We describe in the sidebar below just 
how important improving the lens of problem definition can be—it 
ended up reducing HIV infections by a staggering 600,000 over a 
decade. The team followed the traditional public health (or supply 
side) approach at first, which led to some insights, but balanced 
it with what became a breakthrough effort on the demand or con- 
sumer side, in this case by involving sex workers in developing 
the solution. As a result of early testing of a hypothesis, the team 
was able to agree on a toolkit for replication across India, one that 
built in processes for community engagement and included the 
voices of sex workers that were often not heard. Exhibit 2.4 shows 
another example of “porpoising” between the problem state- 
ment, hypotheses, and the data, to arrive at the most accurate 
problem statement. 
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FRAMING THE PROBLEM 
REDUCING HIV INFECTIONS IN INDIA 


In 2003 the Bill and Melinda Gates Foundation funded the Avahan India AIDS Initiative in 
response to growing concern about the spread of HIV infections in India. Ten years later 
Lancet Global Health researchers estimated the interventions prevented over 600,000 HIV 
infections.! Framing the problem played a major role in the success of the Avahan Initiative 


The Avahan team, led by former McKinsey senior partner Ashok Alexander, reframed the problem 
in terms of supply and demand. They conducted field analysis to understand behavioral and 
causal factors and implemented actions with a strong emphasis on communications at the 
community level 


As Ashok put it: We wanted to strengthen the public health approach (supply). Besides raising 
awareness, we were emphasizing data use by front line health workers—in effect, knowing 
how many condoms to distribute. We were simultaneously making efforts on the demand 
side, enabling workers to come together in groups to campaign for common issues such as 
safety. In business language, this amounted to creating a more active, aware consumer, and 
the demand side gathered momentum.” 


The team then turned to the demand side and asked the sex workers for their views on 
the spread of HIV. They cited all too frequent examples of violence occurring with men 
demanding unprotected sex. The team was able to show a high correlation between violence 
and sexually transmitted infections (STIs). Ashok Alexander and his Avahan team saw this 
as a breakthrough in the work. The team then explored how they could reduce the violence 
perpetrated on the women. Their answer was rapid response teams of community workers, 
often accompanied by a lawyer and reporter from the local newspaper. The approach ‘went 
viral,” as Ashok put it, as communities all over India embraced the intervention piloted in the 
Avahan Initiative 


Bill Gates, in his letter to Warren Buffett in early 20177 wrote, ‘It's well documented that 
the decision of India's sex workers to insist on condom use for their clients kept HIV from 
breaking Into the general population. The Lancet Global Health researchers also pointed 
to “a large scale increase in consistent condom use in high-risk groups since the start of 
the Avahan program. Importantly, they could have also added a commensurate reduction in 
violence against women alongside the huge reduction in HIV infections 


Lancet Global Health article by Michael Pickles et al 
? Personal communication with Ashok Alexander. 
> Gates Notes.com/2017 Annual Letter 


EXHIBIT 2.4 


Porpoising to Get the Problem Statement Right 


Problem statements keep on getting better when facts are brought 
to bear to sharpen the problem definition. We find that setting up a 
dialogue with questions that have to be answered is key to getting 
the required sharpness in problem definition. As an example, Rob 
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was asked to help a team look at the issue of underinvestment in 
capacity building by nonprofits, a key finding of an enquiry by the 
Australian Government Productivity Commission. He asked the fol- 
lowing questions: 

1. Is it true of all or some nonprofits? 

Is it true of large or small organizations? 


Is it likely due to funding available or priorities? 


Is it true for all nonprofit sectors or some? 


an e WW N 


Is the underinvestment in operations effectiveness or managing 
complex systems? 


Some of these questions could be answered with known data but 
others required a perspective gained from interviewing sector 
leaders. The team was able to conclude that the issue of underin- 
vestment was true for most but not all nonprofits; it was particularly 
the case for small organizations with less than fifty employees; it 
was due to the fact that nonprofits had little discretionary funding 
for capacity building; it was more acute in sectors where the service 
delivery model was in question; and, finally, underinvestment was 
especially the case in managing complex systems, even if capacity 
building for operations effectiveness was also underfunded. The 
resulting reworded problem statement went from a very broad 
statement to one that targeted nonprofits that had resourcing dif- 
ficulties to fund capacity building, and a specific need to update 
delivery models for a complex systems environment. Efforts to 
sharpen problem statements can pay off handsomely. In the next 
chapter we show the link between good problem statements and 
good hypotheses to test. You won't be surprised to learn that sharp 
problem statements lead to better and more testable hypotheses. 


Design Thinking and Seven Steps 
One of the most common problem statements we come across 


relates to the user experience. Design thinking has developed as 
a powerful tool for problem solvers tackling consumer needs and 
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the user experience, often in product or service design situations, 
but also outside those areas where creativity in problem solving is 
important to good outcomes. We are often asked where design 
thinking sits in relation to our seven-steps analytic problem solving 
process. We believe it is both consistent and complementary, espe- 
cially in cases where an understanding of the user experience is 
essential. We present cases in the book where the user experience 
is at the core of problem solving and lends itself to design thinking, 
such as our cases on where to live, HIV in India, and whether to have 
knee arthroscopy surgery. 


Design thinking is typically presented as a process, much like the 
seven steps, but with slightly different stages: empathize, define, 
ideate, build, test, and deliver. These six steps are illustrated in 
Exhibit 2.5 below as an iterative and fluid process. While the meth- 
odology is laid out in what seems like a gated, sequential process, 
the steps are usually not linear—they can happen in parallel and 
steps can be iterated and repeated as insight grows. 


There are clear parallels between our steps and design thinking: 
The empathize and define steps correspond closely to the prob- 
lem statement approach; the ideate step is akin to our process to 
break down a problem around hypotheses and then test; and the 
build and deliver steps correspond to our work planning and analy- 
sis steps. As in our approach, design thinking is highly iterative, 
revisiting earlier stages as knowledge accumulates. 


By formalizing the methodology and values of designers, design 
thinking allows teams to use the creative problem solving process 
to find innovative solutions. A user-centered approach is funda- 
mental to this kind of problem solving, where a significant amount 
of time is spent empathizing with and understanding the user and 
their “pain points.” This is done through patient, sustained interac- 
tions over time, including interviewing, observing, and researching. 
We like the idea of integrating design thinking into the seven-steps 
approach, especially when novel and creative ideas are critical to 
the solution. 


Design Thinking and Seven Steps 


OESION THINKing 


IDEATE 


Brainstorm 
Think loosely and outside the box. 
Be optimistic, open, curious, 
generative, and divergent 


DEFINE 


EMPATHIZE 


Understand ee 
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Gain Insights 
Interviews 
User profiles 
Mind mapping 
Five whys 


DELIVER 


Identify i Revise hypothesis 


a nee 


Work iteratively to understand the 
Pain Point” (problem) 


EXHIBIT 2.5 


Design thinking usually starts with understanding the potential 
user's needs. Thus, the first step of the design thinking method- 
ology is empathizing with the people you want to serve, learning 
what their needs are, recognizing why the product or solution could 
be relevant, and how it will be used within their everyday life. The 
cycle to empathize and then build-test-redefine, and build-test- 
redefine again, is based on an idea that you don’t have to make 
decisions based on historical data and instinct. Rather, decisions 
can evolve, based on evidence from users’ reactions to success- 
ful or failed prototypes. Design thinking methodology that focuses 
on user understanding, research, and iterating with prototypes, is 
a powerful tool to use in conjunction with the seven-steps problem 
solving approach, especially in the consumer/product space. 


45 


46 


Define the Problem 


Putting effort into problem definition pays off. It gives you a smart 
start and an efficient process. Iterative re-cuts of the problem defi- 
nition are highly valuable. And finding ways to challenge problem 
framing and scope will give you greater creativity in your solutions. 


Chapter 2 Takeaways 


e Defining the problem well is the starting point for great problem 
solving. If you don’t get it right, you risk wasting your time and 
having an unhappy decision maker. There is an adage that says 
“a well-defined problem is a problem half solved”; it's worth the 
investment of time upfront. 


e Problem definition is more than just your problem statement; 
you need to know the boundaries of the problem, the timeframe 
for solution, the accuracy required, and any other forces affecting 
the decision. 

e Try a challenging antithesis or counterfactual to test the robust- 
ness of the problem statement. It may be around customer 
behavior, competitor reaction, or regulator stance on an issue. 


e You should iterate and refine your problem statement as you 
porpoise into the data and learn more about the problem 

e Bring creativity into problem definition by widening the aperture, 
relaxing constraints, and adding diversity to your team. 


e Try to add elements of design thinking at this stage, particularly 
by adding active empathy for the user or decision maker. 


Problems to Try on Your Own 


1. Take your number one organization priority for this year and 
write it as a problem statement. Then test it against the criteria 
for a good problem statement. How different would it be if you 
wrote it from the CEO's perspective? Or a competitor? 


2. List the top three issues in your part of the world, such as envi- 
ronmental decline, crime rates, employment, opioid abuse; try 


Problems to Try on Your Own 


preparing a problem statement for one. Set out the facts and 
analysis you need to know about the problem. Spend an hour 
collecting facts through online research and some phone calls. 
Then redo the problem statement as a second cut. 


. Define the problem with your banking relationship. What are the 
pain points in your user experience? How might they be allevi- 
ated through product or process changes? 


. Dream up an unmet consumer need and run it through the 
design thinking cycle. 
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Introduction 


Good problem disaggregation is at the heart of the seven steps 
process. Any problem of real consequence is too complicated 
to solve without breaking it down into logical parts that help us 
understand the drivers or causes of the situation. So this is the 
most important step in problem solving: taking the problem apart 
in a way that helps us see the potential pathways to solve it. At the 
same time, when we can see all the parts clearly, we can determine 
what not to work on, the bits that are either too difficult to change, 
or that don’t impact the problem much. When you get good at 
cleaving problems apart, insights come quickly. The magic comes 
in seeing which type of logic tree makes an elegant solution more 
obvious. It is like a plan to cut a diamond—doing it the right way 
really matters. 


The diamond-cutting analogy is fun. But there is a risk in it. It makes 
the problem of cleaving seem too difficult, like it is a task for experts 
only. In truth, any of us can get started with an online literature 
review of the facets of the problem to generate ideas. This brain- 
storming on the shape and elements of the problem leads to an 
elemental logic tree to begin to cleave the problem open. 


In this chapter we will cover the use of logic trees to disaggregate 
problems, keep track of the parts for analysis, and build insights 
toward solutions. We will introduce several kinds of logic trees 
that are appropriate for earlier or later stages of problem solv- 
ing, or for different kinds of problems, including factor/lever trees, 
deductive logic trees, hypothesis trees, and decision trees. Some 
problems give up their answers in straightforward and intuitive 
ways. Others require more patience to yield insight. This chapter 
will cover the use of cleaving frameworks or heuristics for indi- 
vidual, organization, and social level problems that may help crack 
tougher problems more quickly and elegantly. We will then talk 
about prioritization of problem elements to prune off branches 
of the tree that don’t contribute enough to the answer to spend 
time on. And finally we'll talk about team and solo processes that 
make disaggregation and prioritization easier and less prone to 
decision-making biases. 


Types of Logic Trees: Getting Started 


Types of Logic Trees: Getting Started 


Logic trees are really just structures for seeing the elements of a 
problem in a clear way, and keeping track of different levels of the 
problem, which we liken to trunks, branches, twigs, and leaves. You 
can arrange them from left to right, right to left, or top to bottom— 
whatever makes the elements easier for you to visualize. There are 
lots of ways to do it; in fact, we almost always try two or three alter- 
native disaggregations to discover which yields the most insights. 
Think of a logic tree as a mental model of your problem. Better 
trees have a clearer and more complete logic of relationships link- 
ing the parts to each other, are more comprehensive, and have no 
overlaps—but we'll get to all that. 


Here is a simple example of tree construction: The task of planning 
to build a brick wall can be seen as either a process or as the sum 
of its components—both yield different insights and are helpful for 
visualizing the task of building the wall (see Exhibit 3.1). 
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Exhibit 3.2 shows a number of different tree structures we have 
found useful in practice, and a sense of when they are best 
employed. Early on in the process we usually employ component 
or factor trees, and often work inductively (learning from specific 
cases that illuminate general principles later), to help us define 
basic problem structure. Later, after some iterations with data and 
analysis, we usually move to hypothesis trees, deductive logic 
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trees, or decision trees, depending on the nature of the problem. 
The only rule here is to move when you can from trees with gen- 
eral problem elements to trees that state clear hypotheses to 
test; vague labels do not drive analysis or action—which is the 
whole point. 


At the very beginning, when you are able to state your problem 
clearly but don’t yet have a detailed understanding of the prob- 
lem to provide a vantage point on the solution, it usually makes 
sense to employ the simplest kind of tree, a component or factor 
tree. The goal in building logic trees is to find the levers that help 
us crack our problem, starting with the components that can help 
focus data gathering and eventually move us toward good hypoth- 
eses that can be tested. Components or factors are just the most 
obvious elements that make up a problem, like the bricks and mor- 
tar of our earlier brick wall example. You can usually find enough 
information for a logical first disaggregation with a small amount of 
Internet research and a team brainstorming session. Let’s look at a 
case study to make this clearer. 


Case Study: Saving Pacific Salmon—From Initial 
Component Tree to Refined Hypothesis Tree 


Exhibit 3.3 shows an example of this kind of component or lever 
tree from Charles's work in salmon conservation, first introduced in 
Chapter 2. This tree was developed very early in the process, when 
the foundation he worked for knew it was committed to working 
on the problem, but didn’t have a grasp of all the elements and 
relationships that defined the problem space. 


So the team developed a rudimentary component tree to begin to 
get a handle on the problem. To get to this stage we read every- 
thing we could find about salmon and talked to a number of experts 
in salmon conservation. That involved days of work, not weeks—our 
experience is that you should only do enough initial research to 
generate a first-cut tree, since the tree structure will act as a guide 
to make further research more efficient. 


53 


54 Problem Disaggregation and Prioritization 
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As you can see, this first-cut tree is busy. It helps us see the big levers 
that affect salmon, but it doesn't have much insight coming through 
yet, since it is still at the component level, without clear hypotheses. 
We knew freshwater watershed quality and ocean environmental 
conditions would be important to Pacific salmon abundance and 
diversity, as would be the impact of commercial and other fisheries, 
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hatcheries and fish farms, consumer behavior, and government pol- 
icy. We could see some of the secondary and tertiary layers of the 
problem, but we didn’t have a strong sense of importance or mag- 
nitude of any lever, or which ones our grant funding could affect. So 
we weren't in a position to develop hypotheses to guide our data 
gathering, analysis, and prioritization. But we did have a good list of 
factors affecting salmon, and we did have several layers of interven- 
tions mapped out. This gave the team better direction for the next 
stage of research and relationship building in salmon country. It was 
a good start, but only a start. 


This first stage component tree for Pacific salmon is deficient in a 
number of other ways as well. It doesn’t address the substantial 
regional differences in which factors are important. It doesn’t show 
any sense of the magnitude impact of each lever. And it creates a 
significant confusion in the government policy element, which is 
shown as a separate topic area, rather than as it truly plays out by 
affecting each lever from watershed protections to fisheries to arti- 
ficial propagation. 


It is worth stopping here for a moment to introduce an important 
principle in logic tree construction, the concept of MECE. MECE 
stands for “mutually exclusive, collectively exhaustive.” Because 
this tree confuses or overlaps some of its branches, it isn't MECE. 
It's a mouthful, but it is a really useful concept. 


Trees should have branches that are: 


Mutually The branches of the tree don't overlap, or contain partial 

Exclusive and elements of the same factor or component. This is a little 
hard to get your head around, but it means that the core 
concept of each trunk or branch of the problem is self- 
contained, not spread across several branches. 

Collectively Taken as a whole, your tree contains all of the elements of 

Exhaustive the problem, not just some of them. If you are missing parts, 
you may very well miss the solution to your problem. 


Exhibit 3.4 provides a way of seeing it visually. 
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COLLECTIVELY (CE) NOT COLLECTIVELY 
EXHAUSTIVE EXHAUSTIVE 
MUTUALLY 
EXCLUSIVE 
A B 
(ME) 
4 
NOT MUTUALLY 
EXCLUSIVE A B 6 
EXHIBIT 3.4 


To give an example related to our Truckgear competition case from 
Chapter 1, if you are evaluating retailer performance with a return 
on capital tree, you put existing store sales on a different branch 
than new store sales to be mutually exclusive; to be collectively 
exhaustive you make sure your tree includes all sources of sales, 
including existing stores, new stores, and online sales. If you are 
having trouble getting clear hypotheses out of your tree, check to 
see if it is really MECE. 


Constant iteration is key to making the seven steps work. After 
Charles's team had done a literature review, they met with many 
experts, forging relationships on the ground with conservation 
organizations, government agencies, policy personnel, fishers, and 
First Nations leadership. They commissioned exploratory science 
work with salmon researchers, and were able to refine their salmon 
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tree and transition from a simple, components visualization to a 
hypothesis tree. As you can see from Exhibit 3.5, this tree is bet- 
ter organized and MECE, with more active questions. And it has 
focused the team’s analysis on both specific regions and inter- 
vention types. These were not very specific hypotheses at this 
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stage, but they allowed the team to work with its new partners to 
develop a set of initial grants to push some early outcomes (which 
included a ban on expansion of salmon farming further north in 
British Columbia) and promoted greater Foundation understand- 
ing in what became a 15-year, $275m effort to help preserve Pacific 
salmon. At a later stage, which will be discussed in this chapter, the 
team was able to prune off some lower impact or higher difficulty 
branches of the tree, which helped focus its problem solving on 
high-potential outcomes. 


Deductive Logic Trees 


Deductive logic trees take their name from the process of logical 
deduction. Deductive reasoning is sometimes called top-down 
reasoning, because when we apply it we argue from general rules 
or principles to conclusions via more specific data and assertions. A 
classic kind of deductive statement is structured like this: 


e General statement: All watchmakers need glasses. 
e Specific observation: Sally is a watchmaker. 
e Deductive conclusion: Sally needs glasses. 


Deductive logic trees are constructed similarly, with a problem 
statement that may sometimes be expressed in quantities, and 
branches that are typically logically or mathematically complete, 
so that the components add up to the desired objective of the 
problem statement. We use this kind of tree when we know a lot 
about the logical structure of a problem, and especially when the 
cleaving frame is inherently mathematical, as we saw with the return 
on profit levers tree in Chapter 1. A more sophisticated version of 
this includes the investment costs of different actions, not just the 
impact on profitability. 


Why analyze return on invested capital? Return on invested capital 
(ROIC) signals how well a company can turn investments into profit. 
Arithmetically it is the product of two components: profit margin 
(sometimes called return on sales) and capital turnover (the ratio 
of sales to invested capital). ROIC is useful for comparing firms by 
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understanding the levers affecting the return on company assets, 
irrespective of ownership structure (debt and equity) and excess 
cash. If you are interested in business problems, learn this tree. The 
next case study focuses on the use of a return on capital tree to 
understand the competition between two businesses. You can also 
find a more generic example in Exhibit 3.9. 


Case Study: Battle of the Hardware Kings: Hechinger 
versus Home Depot 


As industries change, business models are tested through tough 
competition and market disruption. Through the use of a ROIC tree, 
Charles was able to determine which levers were driving the prof- 
itability and growth of two competitors in the hardware industry. 
Charles worked for a consulting company that was hired by Hech- 
inger Hardware to develop an expansion plan into new markets. 
The problem definition step revealed that in order to successfully 
expand, the business had to be able to withstand the competition 
of its rising peers. This tree helped the team he was on discover 
that an epic battle was brewing. Here is the story. 


Hechinger Hardware opened its first hardware store in 1919, focus- 
ing its marketing strategy on the retail consumer, rather than con- 
tractors, fueling the growth of the do-it-yourself hardware business. 
While it had begun slowly expanding across its first half century, 
the family-owned company planned for more rapid expansion after 
going public in 1972. At that time the hardware industry was grow- 
ing at nearly 12% annually, more than two times the US economy, 
and rising home prices increased homeowners’ attention to upkeep 
and continuous improvement of their homes. 


The Company [Hechinger] has an enviable track record, includ- 
ing 39 consecutive quarters of positive earnings per share 
comparisons ... 


Alex Brown and Sons Analyst Report, FY1984 


With strong performance and promising industry predictions, 
Hechinger was looking to expand into new markets. It had 
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$130 million of cash available for expansion on its books and strong 
employee seniority—150 managers across the chain had nine years 
or more of experience—so company management felt confident in 
their ability to build market share. 


However, the consulting team’s analysis uncovered a problem for 
Hechinger that it didn’t see coming. A small southern competi- 
tor, Home Depot, had opened three new warehouse-style stores 
starting in 1979, a novel idea. In contrast to Hechinger’s spotless 
stores, extensive backrooms for inventory storage, and relatively 
high prices, Home Depot built a warehouse hardware superstore 
model intended for both retail shoppers and contractors. Going 
public in 1981 they began to rapidly expand from their Atlanta 
base, coincident with Hechinger’s expansion from Washington, DC, 
setting up a showdown. While Hechinger undoubtedly had excel- 
lent management and a strong record of historical success, at the 
time they didn't see that Home Depot was changing the industry's 
competitive dynamic. 


Faced with a competitive player in a number of promising expan- 
sion markets, Charles and his team set to work with a comparative 
analysis of Hechinger and Home Depot and were startled by what 
they found: What appeared to be similar businesses externally were 
radically different internally. To do this problem solving the team 
employed a return on invested capital tree, which turned out to 
be a powerful problem solving tool. What it showed is that while 
return on capital was similar for the two businesses, the drivers of 
success were different. Using publicly available accounting data, 
Exhibit 3.6 shows: 


e Home Depot had lower overheads and lower costs of goods 
sold, but it also had lower gross margins (selling price minus cost 
of goods)—it was following a low-cost/low-price strategy. 

e Home Depot had much higher asset productivity, meaning it 
generated more sales per dollar invested, including higher sales 
per square foot of store space and higher inventory turns. 


e Home Depot was growing much faster, although it was of similar 
size in 1983. 
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RETURN ON INVESTED CAPITAL 


HECHINGER VS. HOME DEPOT 


Gross profit < 7 


Gross margins = 


1983 033 


027 


1988 
Return on sales = 


_ Operating income 


Net sales 
1983 rsa 
P Ea Selling, General, & 
__Admin Expenses _ 
Net sales 
1983 023 
{ta 
1988 020 
RETURN ON 
A 
INVESTED 
CAPITAL (ROIO 
1983 038 
E 
1988 019 
Fixed Assets 
Co = 
Net sales 
1983 mon 
on 
1988 08 
ia 
Capital turnover = + 
m | e Working capital 
Net sales 7 t "Net sales 
oe + 
Invested capital 1983 029 
1983 373 (cis 
ms 1988. 029 
1988 | 239 oor 
isi] 
_Intangibles = cash 
Net sales 
1983 -04 
1-008 
Data sourced from Compustat (1983, 1988) 
1988 -006 


1 Alex, Brown, and Sons (1984 & 1985) 
? The first Boston Corporation (1987) 


EXHIBIT 3.6 
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Clearly, Home Depot was making up for lower margins by having 
lower overhead costs and higher asset productivity, generating a 
similar return to the long-established Hechinger. The team decided 
to dig deeper. What it discovered was remarkable. Home Depot 
was growing so quickly because it had much lower prices. Charles's 
team conducted secret shopper pricing analysis and found 
that Home Depot's prices were 15 to 18% lower for comparable 
products. How could it possibly operate at such low prices? 


The ROIC tree was the central framework to guide the analysis. What 
the team learned was that Home Depot had built a state of the art 
inventory management system, and had developed a store design 
that housed inventory on 20-foot racks on the retail floor (versus 
Hechinger's 8-foot racks), therefore doing without backroom stor- 
age and labor intensive restocking. Moreover, instead of storing 
goods mainly in a central company warehouse, it negotiated with 
suppliers to direct ship to Home Depot stores, or cross-dock goods 
(a procedure in which a truck from a supplier is quickly unloaded 
and the goods go straight to store delivery trucks without being 
double handled). The team estimated that Home Depot was able 
to operate with 5-7% less shipping and logistics costs than Hech- 
inger, and negotiate as much as 5% lower costs for the same goods 
by concentrating volume on orders of fewer retail iterns (sometimes 
called stock keeping units, or SKUs). 


Cross-docking or breaking up manufacturers’ shipments to go to 
the various stores is undertaken on a small scale, but for the most 
part, suppliers are more than willing to drop ship to the stores 
because the individual locations order by the trailer load. Home 
Depot's average sales per store run around $20 million, or more 
than double Hechinger’s or Builders Square's. 


Analyst report, The First Boston Corporation 


So Home Depot was generating the same return on invested capi- 
tal, but able to make up for much lower prices with a combination of 
lower distribution costs, cheaper sourcing, lower store and central 
overheads, and much higher asset productivity. Lower prices were 
driving much higher sales per store and a much higher company 
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growth rate. Trouble had quickly arrived for Hechinger: Could it 
adopt some of Home Depot's competitive practices and regain its 
regional lead based on its strong brand? 


Although Hechinger tried to lower its costs and increase its asset 
productivity, over time the differences magnified, and by 1988 the 
businesses had diverged (see Exhibit 3.7). Home Depot's sales 
were accelerating as it opened new markets and drove nearly 
twice the sales per store than Hechinger, and its asset productivity 
continued to climb. 


Inventory Turns 
6.5 


1982 1986 1990 1994 


Inventory Turns-- Hechinger == Inventory Turns-- Home Depot 


EXHIBIT 3.7 


As Hechinger tried to compete with Home Depot's prices, they 
offered sales and promotions, but—despite the acquisition of 
Home Quarters, a similar warehouse chain—the company failed to 
make the deep business model changes required to compete. With 
new entry on many fronts across 19 states, including their long- 
time, dominant Washington, DC home market, they began to close 
stores. By 1997 Hechinger was out of business! (see Exhibit 3.8). 


‘Margaret Webb Pressler, “The Fall of the House of Hechinger,” Washington Post, 
July 21, 1997. 
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Sales & Operating Income: Hechinger Vs. Home 
Depot, 1983-1996 
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D Q D A & V 
PPP PP SSH 
EXHIBIT 3.8 


The return on capital tree shows the fundamental differences 
in business models that led to this competitive result. It demon- 
strates the value of a component disaggregation problem solving 
approach applied in a business setting. 


Exhibit 3.9 is a more generalized version of a return on invested 
capital tree. We have expanded several branches out to highlight 
the levers that impact the return on capital metric that might help 
you with your other business problems. 


In the hardware company comparison, the objective of the tree 
was return on invested capital, and the branches included prof- 
itability and asset productivity, drivers that mathematically led to 
the difference in operating and financial results between competi- 
tors Home Depot and Hechinger. This helped us understand the 
two companies’ prospects in the market by highlighting how their 
two different business models ultimately led to different levels of 
financial success. (One thing to be careful about in using return 
on capital trees is that the definition of capital and profit typically 
employed is an accounting one (so subject to accounting rules on 
things like depreciation), rather than a cash or replacement cost 
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definition. You may benefit from cross-checking your numbers with 
a replacement cost calculation for core assets like stores and using 
cash profitability.) 


Case Study: Bay Area Nursing Related Patient Outcomes 


To show the clarity deductive logic trees can bring, let's introduce 
a new example, this time around improving nursing in hospitals in 
the San Francisco Bay Area. Charles observed a team working on 
this problem at the foundation he worked with. The team wanted 
to improve nursing-related patient outcomes in the Bay Area in 
California. Nurses provide upwards of 90% of patient care in hos- 
pitals. More than 100,000 lives a year are lost in the United States 
from mistakes in patient care from many sources, especially from 
sepsis, ventilator acquired pneumonia, hospital acquired infections, 
and medication errors. There is a substantial shortage of nurses 
in California and beyond, and studies show that for each patient 
added per registered nurse (RN), mortality rates increase. Every 
one of us has a lot to gain from making progress on this problem. 


Early on in the project the team constructed a relatively straightfor- 
ward deductive logic tree focused on increasing both the number 
of skilled nursing graduates, and on improving the skills and prac- 
tices of existing RNs. 


By developing this deductive logic tree, shown as Exhibit 3.10, and 
more sophisticated versions that followed, the foundation nursing 
team was able to focus its attention on the key drivers of nursing 
numbers and skill levels. This allowed the team to use data and 
analysis to determine which levers were most powerful in improving 
patient outcomes, and which were cost-effective to address. They 
developed a series of strategies to address these levers, including: 


e Increasing the workforce of well-trained nurses via new nurs- 
ing schools 


e Implementing evidence-based nursing practices in hospitals 
e Supporting frontline nursing leadership 
e Improving patient transitions from hospitals 
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IMPROVING NURSING OUTCOMES 


DEDUCTIVE LOGIC TREE 
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Increase interest in 


nursing in high schools 


Retain more nursing 
students in programs 


ncrease funded places 
in training programs 


mprove nursing school 
curriculum to best practice 


Upgrade skills of 
nursing faculty 


Upgrade nursing protocols 
to best practice 


mprove work practices 
of entire care teams 


mprove frontline nursing 
eadership support 


Fund advanced training 
rograms in nursing schools 


Create on-the-job skills 
training programs 


After 12 years of investment in this challenging problem, the 
foundation's grants helped substantially increase the nursing sup- 


ply in the Bay Area, with more than 4,500 


RNs added, nursing 


school curriculums improved, bloodstream infections and readmis- 
sions rates reduced, and an estimated 1,000 lives saved a year from 


sepsis. Good problem solving matters. 


Inductive Logic Trees 


Inductive logic trees are the reverse of deductive logic trees. They 
take their name from the process of inductive reasoning, which 
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works from specific observations toward general principles. Here is 
an example of inductive reasoning: 


e Observation: Sally is a watchmaker and wears glasses. 

e Observation: Shaun is a watchmaker and wears glasses. 
e Observation: Steven is a watchmaker and wears glasses. 
e Inductive assertion: Watchmakers typically wear glasses. 


We employ inductive logic trees when we do not yet know much 
about the general principles behind the problems we are inter- 
ested in, but we do have some data or insights into specific cases. 
Inductive trees show probabilistic relationships, not causal ones: 
Just because all the swans you have seen are white, does not mean 
all swans are white (there are black swans in Australia.’) In many 
cases you will actually work on your initial tree both inductively and 
deductively. You will have a sense of some of the bigger drivers 
and general principles, and you will have good case examples of 
successful projects in the space you are looking at. So you work 
both from the trunks of the tree and from the leaves, slowly and 
iteratively figuring out which is which. 


Case Study: Dealing with Contested Historical 
Legacies—from Inductive Thinking to a Decision Tree 


As we have seen in recent news stories focused on controversial 
statues and public monuments, there has been a lot of public inter- 
est in historical artifacts that chronicle or memorialize historical lead- 
ers whose views are out of step with current values, initially sparked 
by activism on university campuses, but now including municipali- 
ties and other institutions. As the head of the Rhodes Trust, one of 
many institutions with a complicated historical legacy, Charles had 
been grappling with these issues. 


What Charles observed is a lot of ad hoc approaches to each his- 
torical artifact. Under pressure from interest groups and external 
commentators, institutional decision makers have been lurching 
from decision to decision, and often reversing course, with no 


?See Nassim N. Taleb, The Black Swan: The Impact of the Highly Improbable (New 
York: Random House, 2007), for a deeply insightful discussion of probability and 
decision-making errors. 


Types of Logic Trees: Getting Started 


clear set of principles to guide their actions. Good moral reasoning 
requires, well, reasons, and Charles thought that the debate would 
be improved by developing some principles for how to deal with 
artifacts of historical characters. This is complicated by the fact that 
even our favorite historical characters held views that we would find 
abhorrent today. So the framework had to accommodate heroes, 
such as Frederick Douglass, Thomas Jefferson, Walt Whitman, and 
Gandhi, not just historical bad guys. 


This turns out to be an important insight. The team began an 
inductive-reasoning exercise where we took a large range of histori- 
cal characters about which there is rich information and even some 
agreement on how to assess them. For example, we compared 
Thomas Jefferson with the reputations of other presidents who 
owned slaves, and we looked at the legacies as varied as John C. 
Calhoun at Yale, to Aulus Plautius, the Roman general who invaded 
Britain in 43AD. By working backwards from how people think about 
individuals, the team began to tease out some general principles of 
judgment. At first all they had was a list of threshold questions that 
underpinned the judgments or assessments, but no clear hierarchy 
of what questions were most important, as seen in Exhibit 3.11. 


ARTIFACTS AND CONTESTED CHARACTERS: 
THRESHOLD QUESTIONS 


Is the wrong Were the ideas 
committed the contested at the 
principal legacy time or consistent 
of the contested with values of the 

persone day? 
Does the artifact 
chronicle or 
memorialize? 


Can we identify 
those harmed? 


What is the scale Were the wrongs 
of the harm? beliefs or acts? 


Is there ongoing 
harm or benefite 


EXHIBIT 3.11 
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After some intensive whiteboard sessions to figure out the hierarchy 
or order of the principles, the team came up with the following 
decision tree for how to address these artifacts of difficult historical 
legacies (see Exhibit 3.12). 


ARTIFACTS AND CONTESTED CHARACTERS: 
DEALING WITH DIFFICULT HISTORICAL LEGACIES 


Range of Potential Remedies 
Does the artifact honor the OEA 
contested actor or simply chronicle? ————— > Study, acknowledge 


Honor Single good act 


(e.g. benefaction) 


Does the artifact honor the whole —— > Contextualize/Placard the legacy 
person or a single good act? 


v 


Whole person 


RA Beliefs 
Are the wrongs beliefs or acts? >  Contextualize/Placard the legacy 


Acts | 
No 


Were the acts substantially contested -——————————_>_ Contextualize/Placard, Balance 
in the historic period of the actor? 


Yes 
No 
Can we identify victims or direct >  Contextualize/Placard, Balance, 
descendants? Reshape, or Edit 
Yes 
No Contextualize/Placard, Balance, 
Can we identify documented harm and --——————————_»__ Reshape, or Edit, depending on 
is it ongoing? level of association with the 
institution 
Yes 
z No 
Is it at a large scale? > Contextualize/Placard, Balance, 
or Rename 
Yes 
4 No Moral balancing, and then 
Is it the primary legacy of the actor? ——_> Contextualize/Placard, Balance, 
Reshape, Edit, or Rename 
Yes 
Contextualize/Placard, Rename, 
» Remove, depending on level of 


association with the institution 


EXHIBIT 3.12 


Step 3: Prioritization—Pruning Your Logic Trees 


You may agree or disagree with the threshold questions and range 
of sanctions the team came up with,? but hopefully you find use- 
ful the idea of brainstorming some pragmatic principles by a 
combination of inductive reasoning about contested individuals 
and reference to historical philosophical maxims, and then using 
this output to generate a decision tree to guide action in a more 
systematic way. 


Step 3: Prioritization—Pruning Your Logic Trees 


Good problem solving is just as much about what you don’t do 
as what you do, and good prioritization in problem solving makes 
solutions come faster and with less effort. We want our initial trees 
to be MECE, so that we know we have all the parts. But we don’t 
want to retain elements of the disaggregation that have only a 
small influence on the problem, or that are difficult or impossible to 
affect. Before we start to invest significant time and effort into work 
planning and analysis, we have to prune our trees. 


We have found the 2x2 matrix in Exhibit 3.13 to be a useful forcing 
device for prioritization of which levers to work on, here shown for 
the Pacific salmon case study. The left-hand axis is whether or not 
your efforts could have an impact on the problem lever or branch. 


3Remedy Definitions 


Contextualize/Placard: Leaves artifact in place, but tells the story of the actor's role 
in offensive acts. 


Balancing: Brings into the physical or digital space the voices/images of others, 
including those wronged, may include balancing restorative justice actions. 


Relocation: Removes the artifact, but to a place it can be studied. 


Reshaping or editing: Original monument is left in place, but altered in a way to 
highlight offensive aspects. 


Destroying or removing (as it sounds). 
Renaming (as it sounds). 


Moral Balancing: Applying a moral calculus to determine the principal legacy of 
the figure. 


Reparations and/or restorative justice. 
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PRIORITIZATION 
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EXHIBIT 3.13 


The bottom axis is how important that lever is in affecting the overall 
problem you are trying to solve, in this case the impact on salmon 
abundance and diversity. 


As you can see from the matrix, some prioritization decisions are 
relatively straightforward: Even with the resources of a large foun- 
dation, our team couldn't hope to influence climactic conditions, 
which are the main driver of ocean productivity—providing the 
food for juvenile salmon at sea. Although this lever has a significant 
impact on salmon diversity and abundance, we have to lop it off. 


Second, the impact of hatcheries and “ocean ranching” (putting 
artificially raised salmon into the ocean hoping they will return to be 
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captured) is not insignificant—these fish compete with wild fish for 
food, and may cause genetic introgression via inbreeding—but in 
the initial cut at the problem we left it off our priority actions. While 
there is little evidence of net supplementation or cost-effective 
returns, hatcheries are big job providers and have huge support 
from indigenous groups and sports and commercial fishers. It is a 
tough lever to move. 


As you will see in Exhibit 3.13, the team also did not prioritize 
investments to repair damaged ecosystems in the southern end of 
the salmon’s range. We agonized over this decision, but the cost 
of restoration compared to the costs of protection of more pris- 
tine northern systems was not favorable. And the growing impact 
of climate change on water temperatures in these southern systems 
meant there was no guarantee of durability of any results. 


In the spirit of iterating steps, it may also be valuable to revisit the 
problem context to see if relaxing constraints in the problem state- 
ment would make a difference to which branches you prune. As the 
Pacific salmon project progressed through its 15-year life, we did 
make targeted investments in limiting the spread and impact of 
hatcheries, although it never became a major focus. In general, 
problem definitions with tight constraints on the solution space yield 
answers more quickly, but sometimes at the expense of creativity. 
One approach is to relax each constraint and see if the solution set 
opens up in a substantial enough way to be worth the extra work, 
time, and investment required to make those options viable.* 


Advanced Class: Using Cleaving Frames 
to Take Apart Problems 


In the previous sections we explored different types of logic 
trees, the basic structures for problem disaggregation, and 
problem prioritization. We addressed cases early on in problem 


“J. Hammond, R. Keeney, and H. Raiffa, Smart Choices (Broadway Books, 
1999), 48-49, 
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solving where we only knew enough to break problems into basic 
factors or levers, or where we had to work inductively from spe- 
cific cases to move toward general principles of understanding. 
And we examined problems where we had enough information 
to drive from specific hypotheses, where we could use deduc- 
tive logic trees and decision trees. As we saw, there is no abso- 
lute right or wrong way to construct trees—aside from aiming 
for MECE trees—and tree types often overlap. Hypothesis trees 
can be deductive in logic, and inductive reasoning can generate 
effective decision trees. 


Now we are going to take the conversation to the next level. Expert 
practitioners of problem solving often employ existing frameworks 
or theories to more quickly and elegantly cleave problems into 
insightful parts. We have seen an example of this already with the 
hardware company competitive battle case study that employed a 
particular kind of deductive logic tree, a return on capital tree. In 
fact many business problems benefit from this particular type of 
problem disaggregation, because it shows the levers of revenue 
(price, volume, market share), costs (fixed, variable, overheads), and 
asset utilization so clearly, and in mathematical relationship to each 
other. This kind of tree also makes “what if” competitive scenario 
analysis easy. For example, it is easy to model both niche and broad 
market product strategies, and for your team to debate how realis- 
tic the assumptions are that generate results. We often call this kind 
of work “what you have to believe” analysis. 


Just as return on capital trees are often an incredibly insightful type 
of problem disaggregation for business problems, other theoreti- 
cal constructs or frameworks can be powerful cleaving tools for 
other kinds of problems. Exhibit 3.14 shows a toolkit of what we 
call cleaving frames for different kinds of problem. Good problem 
solvers have toolkits like this that act as lenses to visualize poten- 
tial solutions. They try on one or more theoretical frames to see 
which one is likely to be the best fit for the problem at hand. Often 
they combine more than one frame to make progress on particular 
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CLEAVING FRAMES 


BUSINESS 


Frame 


» Price/Volume 


» Market/Ability to 
compete 


Elements 


arket share, awareness, trial, 
repurchase, elasticities 


» Where to play, how to fight, 
acquiring a reputation, 
signalling 


» Opportunity, competitive 
osition, capabilities, resources 


» Growth/Share, explore vs 
exploit, disruptive entrants 


Example 


» Product entry strategies 


» Any competitive situations 


» CSIRO Wifi 


» BHP 


» BHP 


turns 


» Return to capital, valuing 
options 


» Business model construction 
» Home Depot—Hechinger 


» Rent, own, borrow, share, 
zero-basing 


» Asset efficiency 


» Uber, Airbnb 


» Aligning incentives, 
monitoring 


» Compensation, insurance, 
second-hand markets 


» Valuing potential (real and 
potential), puts/calls 


games 


» Customer! 
Shareholder 


EXHIBIT 3.14 


problems. Let's spend some time highlighting a few of these to 
make this notion clearer. 


Continuing with the focus on business problems for a moment, many 
company performance and competition questions benefit from a 
series of frames that help highlight the likely solution paths quickly. 
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This list isn't meant to be exhaustive, as many problems combine 
elements from more than one frame. Some example frames: 


e Price/Volume: One of the key elements of our return on capital 
tree is the revenue drivers branch that focuses on product pric- 
ing and volume. This frame raises questions about the nature 
of the competitive game: Are there differentiated products or 
commodities? Are there competitive markets or oligopoly mar- 
kets controlled by a few players? Each has different dynamics 
and good business problem solvers build these into their disag- 
gregation and research plans. The kinds of elements here often 
include assumptions about market share, new product entry, rate 
of adoption, and price and income elasticities. 


e Principal/Agent: Principal/agent problems occur where certain 
activities are done by agents (think contractors or employees) on 
behalf of principals (think investors or managers), and are very 
common in business. At the core of this kind of problem is the 
need to create incentive structures to align the interest of the 
principal and the agent, when the principal has incomplete vis- 
ibility or control over the work of the agent. The best structures 
provide good, checkable results for the investor and fair incomes 
for the contractor. It sounds easy but it isn’t. This kind of prob- 
lem appears not just in contracting situations, but also in any 
second-hand asset sale and in any insurance problem. 


e Assets/Options: Every asset a company controls, or could pur- 
chase, creates options for future strategic moves that may have 
substantial value. Rob worked for an Australian mining company 
that asked his team to evaluate an investment in a particular 
property. What the team discovered was that the investment in 
the port and the railway infrastructure to exploit that particular 
property created hugely valuable options for a whole range of 
other sites. It made his client the “natural owner” (a powerful 
asset/option concept developed by Rob and another colleague 
John Stuckey) for other sites worth billions of dollars in option 
value. This case is outlined in more detail in Chapter 8. 


e Collaborate/Compete: Any business strategy needs to take 
account of the potential reaction by rival firms. Each company 
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needs to decide where it is willing to engage in intense competi- 
tion (say based on pricing or large investments) and other cases 
where it is not, either because of the nature of the market or the 
nature of the other competitors. The tools and elements in this 
frame are those from game theory, and include concepts such 
as multi-play games and reputation. Our CSIRO Wifi example, 
shown later, employs this frame to decide whether to go to court 
to protect intellectual property. 


At the level of societal problem solving, in the policy space, there 
are many other useful frames, shown here in Exhibit 3.15, most of 
which will be familiar to you. 


CLEAVING FRAMES 


SOCIAL / CITIZEN 


Frame 


» Demand/Supply 


Elements 


» Can we get more? Use less 


Example 


» Water, CO, energy, traffic 


» Social risk; type, level, and risk 
of harm 


flu 


» Tax, tax credits, capital gains, 
ownership 


» Reduce harm, address harm, 
resilience 


» Environment, climate change, 
violence, drug use 


» Market design, property rights, 


regulatory design 


» Cap and Trade vs. carbon 
tax, water rights, in-stream 
reservation 


» Community needs, individual 
rights 


EXHIBIT 3.15 


e Regulate/Incent: Policy makers often face the choice of adding 
legal regulation to address a problem, or using taxation, subsi- 
dies, or nudging policies that provide incentives for people or 
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firms to adopt the desired behavior. This frame frequently comes 
into play around pollution and other externalities. 


e Equality/Liberty: Many policy decisions to address social prob- 
lems face the fundamental frame choice of encouraging more 
equality among citizens, versus allowing more individual free- 
dom. All policies that involve taxing/spending have elements of 
this frame. 


e Mitigate/Adapt: This frame contrasts policy efforts to reduce 
harm from some causal factor, with efforts to adapt to the factor. 
This is often the axis in the climate change debate. 


e Supply/Demand: This frame addresses questions such as “Can 
we get more?” versus “How can we use less?” and is often the 
lens employed for dealing with problems like water supply, health 
care, and energy. 


Case Study: Climate Change Abatement and the 
Cost Curve 


Climate change is a particularly thorny problem and often is cleaved 
around mitigation and adaptation, or incidence and severity (think 
of Kiribati less than a meter above the ocean!). The cleave Charles 
and Rob think is particularly powerful is along demand/supply lines 
where the returns from, or costs of, abating CO, are shown by the 
type of activity and the tons abated. This is called a cost curve, and 
it reads from left (lower cost) to right (higher costs).° 


What you see very quickly in Exhibit 3.16 is that there are a large 
number of abatement activities for which there are positive returns 
to private companies and individuals. This means that a huge 
amount of CO, reduction can occur with good education efforts 
and perhaps supporting tax credits for the investment costs. There 
is a second class of actions, mostly in the agriculture and land use 
space (reforestation, degraded land recovery, avoided deforesta- 
tion), where the investment costs are relatively small compared to 


SMcKinsey & Company, Pathways to a Low-Carbon Economy: Version 2 of the 
Global Greenhouse Gas Abatement Cost Curve, September 2013. 
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the CO, reduction benefits. The third bucket captures actions that 
are longer term and that will require substantial private and social 
investment in new technology and markets. In our opinion, this cleav- 
ing frame provides a useful counterpoint to the mitigate/adapt lens 
on the problem. When we look only at individual (see Exhibit 3.17) 
or corporate carbon footprints, we often miss the opportunity to 
aggregate actions across players and achieve larger results. 


CLEAVING THE PROBLEM 
USING SUPPLY AND DEMAND 


Global GHG Abatement Cost Curve beyond business as usual-2013 


Gas plant CCS retrofit 


Abatement cost Coal CCS retrofit 
€ per tCO,e Iron and steel CCS new build 
60 Low penetration wind — Coal CCS new bulld 


Retrofit residential HVAC 


r Tillage and residue mgmt 
r Insulation retrofit (residential) 
Cars full hybrid 


Pastureland afforestation - 
Degraded land restoration ~) 
2 generation biofuels 

Building efficiency 

new build 


Cars plug-in hybrid Power plant biomass 
Residential electronics Degraded forest reforestation — co-firing 
Resigential appliances Nuclear 


r Waste recycling 


gor 


` Oroanie soll restoration 
L Geothermal 
= Grassland management 
L Reduced pastureland conversion 
Reduced slash and burn agriculture conversion 
Small hydro 
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Rice management 
“ Efficiency improvements other industry 
Electricity from landfill gas 
= Clinker substitution by fly ash 
© Cropland nutrient management 
Motor systems efficiency 
Ł Insulation retrofit (commercial) 
Lighting — switch incandescent to LED (residential) 


solutions and 


Reduced intensive 
agriculture conversion l 
High penetration wind 
r! 
! Solar PV i 
Solar CSP 
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Largely nature’s 


Abatement potential 
GtCO,e per year 


Invest in new 
technology and 
markets 


a agricultural practices 
Just do it now- 


It makes sense! 


EXHIBIT 3.16 
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CLEAVING FRAMES 


INDIVIDUAL 
Frame Elements Example 
» Work/Play » Leisure vs. effort » Choosing a career 
» Weight loss 
» Education 
» Near TermiLong term » Spend vs. invest » Solar panels 


» Retirement savings 


EXHIBIT 3.17 


Problem solving in the individual or personal space can also benefit 
from cleaving frames. Many of the frames we employ in businesses 
around investment decisions work equally well for personal deci- 
sions, such as Rob's solar panel decision. Other frames useful for 
personal problem solving include: 


e Work/Play: How much/how long to work set against non-work 
interests (for example, when to retire and whether your savings 
are adequate). 


e Near term/Long term: Another related category of personal 
decisions focuses on trading off relatively certain near term 
decisions (for example, investing in education) against longer- 
term and more uncertain future possibilities (what skills will be 
required in two decades’). 


e Financial/Non-Financial: Another cleaving point around deci- 
sions like moving to a new house or changing jobs is those 
elements of the decision that are financial (how much do houses 
cost in the new city?) versus those that are non-financial (are the 
schools good enough’). 
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The most powerful way to reach a useful disaggregation of a prob- 
lem is to start with a particular cleaving frame and then do some 
back of the envelope calculations to confirm that it would provide 
insight, as the CO, abatement cost curve so clearly does to support 
a supply/demand split. 


In an example of this approach, Rob was arguing to his team at 
The Nature Conservancy Australia (TNCA) that they should focus 
on Western Sydney to assess the role of urban green spaces in 
asthma hospitalizations. His hypothesis, based on some early 
comparative data, was that parks and other green spaces led to 
lower incidence and severity of asthma. He decided to use the 
incidence/severity cleaving initially for no other reason than it was 
a familiar frame for accident and injury reporting on other boards 
he has served on. He had to make the case to the team that this cut 
on the asthma problem would be insightful. With available data he 
was able to show that Western Sydney warranted attention for the 
following reasons:® 


1. The incidence of asthma was 10% higher for Western Sydney 
than Northern Sydney for adults 16 years and over. Australia has 
among the highest levels of asthma in the OECD. 


2. The severity of asthma, measured by the hospitalization rate was 
65% higher in Western Sydney and the death rate from asthma 
54% higher—so incidence is higher, but severity is much higher. 


3. Evidence pointed to a lack of urban green spaces as a poten- 
tial driver of severity. Tree cover in Western Sydney was esti- 
mated at 15-20% compared to 30-40% in Northern Sydney. In 
addition, particulate matter levels (PM2.5 maximum) are 54% 
higher in Western Sydney. Tree respiration appears to play a role 
in absorbing particulate matter, a topic we illustrate in Chapter 6 
in the case of London's air quality. 


6Health ata Glance 2017:O0ECD Indicators, Health Stats NSW, EPA NSW, iTree assess- 
ment of canopy cover, Figure 6, UTS, Institute for Sustainable Futures, May 2014. 
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As a result, the team is developing an experimental design to sys- 
tematically explore the green space issues raised by the incidence/ 
severity cleaving. 


Again, the point in all of this discussion of using theoretical frame- 
works or concepts to aid problem disaggregation is to find a cut 
through your problem that exposes insight. Different frames often 
yield different insights, so it makes sense to test several. And you 
can often try a social frame on a business problem, or vice versa. 
While it is always impressive to watch experienced problem solvers 
quickly disaggregate difficult problems into insightful chunks, there 
is also arisk in assuming “I have seen this one before, it is an X prob- 
lem.” We often see the world via mental models (another term for 
frames) that have worked for us before. When we encounter genu- 
inely novel problems, we sometimes persist in using frames that are 
unhelpful or misleading in the new and different context. For cen- 
turies, astronomers thought the sun revolved around the earth, and 
scratched their heads when important navigation problems weren't 
easy to solve! The next section addresses team processes that can 
help speed problem solving while avoiding the potholes of false 
pattern recognition. 


Team Processes in Problem Disaggregation 
and Prioritization 


These problem solving steps benefit substantially from team- 
work, and solo practitioners should consider enlisting family or 
friends to assist them here. Because it is often difficult to see the 
structure of the problem, team brainstorming is hugely valuable, 
especially when trying different lenses or cleaving frames. One 
technique we have both found valuable is to use large yellow 
Post-it notes to capture team member ideas on early logic tree 
elements, as shown in Exhibit 3.18. These can then be moved 
around on a whiteboard as the order and hierarchy of trunks, 
branches, twigs, and leaves becomes clearer. 
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EXHIBIT 3.18 


While it is a good idea to consult the literature and look for best 
practices around your problem, it is not uncommon for individu- 
als to get attached to their initial cut through the problem, and as 
mentioned earlier, reliance on external views creates a risk of apply- 
ing incorrect frames. This is even—or especially—true of experts. 
Use of constructive challenging and “what you'd have to believe” 
questions can help get the process out of ruts, and foster more 
creativity in solution paths. Team brainstorming sessions can really 
help avoid attachment in problem prioritization. Another tech- 
nique we have used is giving every team member 10 sticky notes 


and let them allocate them in a voting process for priority analyses 
(see Exhibit 3.19). 
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VOTING PROCESS 
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EXHIBIT 3.19 


Problem disaggregation is central to good problem solving. We have 
illustrated the different type of logic trees that you can use when you 
have a problem to solve. We encourage getting started with a tree 
and filling it out with what you know already. We have shown the 
improvements you can make with further iterations of the tree. 


Your creativity is often likely to come from how you cleave a prob- 
lem. Cleaving brings elements of art and science to your problem 
solving as we show by way of example. In reflecting on how we 
chose to cleave a problem in a particular way, we have been able to 
develop a comprehensive list of ways to cleave business problems 
and those encountered in life. 


Problems to Try on Your Own 


Chapter 3 Takeaways 


e Problem disaggregation is critical to good problem solving; it 
gives us manageable chunks to work on and allows us to begin 
to see the structure of the problem. 


e Start with simple factor or component trees when you are just 
starting out and don’t know much; later, work toward more com- 
plete deductive logic trees and decision trees. 


e It often helps to solve trees backwards, inductively, when you 
know more about the detailed issues (think of these as the leaves 
on the tree) than you do about root causes. 


e See if you can make your tree structures both mutually exclusive 
(no overlapping branches) and collectively exhaustive (no miss- 
ing branches). 


e For business problems it almost always makes sense to start with 
a profit lever or return on capital tree. 


e As you get more experienced, try the broader set of cleaving 
frames from the later part of the chapter; different slices through 
the problem yield different insights. 


e Prioritization is really important once you have a good tree struc- 
ture; try putting all your twigs and leaves on a matrix of the size of 
potential impact and whether you can affect it. Focus your early 
efforts on the big levers that you can pull! 


Problems to Try on Your Own 


1. Take a growth opportunity for your business or nonprofit and 
prepare a logic tree to make the case, first using inductive logic 
then deductive logic. 


2. Try creating a logic tree for the growth opportunity expressed in 
arithmetic relationships. 
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3. Set up a priorities matrix for the top half dozen priorities in your 
business plan. Which ones should you ramp up with resources 
and slow down or terminate based on your assessment? 


4. How else might you cleave the nursing supply problem? 


5. Would you cleave the airport capacity problem in a more com- 
pelling way than outlined in Chapter 1? 


Chapter Four 


Build a Great Workplan 
and Team Processes 


A 
wort? 


a 


Solving some problems is easy. All you need is a piece of paper 
or a whiteboard, a calculator, and an Internet connection to get 
data or access expert opinion. The Where-to-Live problem we 
presented in Chapter 1 is pretty much like this. You have a family 
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meeting to talk about what characteristics in a town are important 
to each family member, you agree on some rough weightings 
(Is trail running availability more important than access to fly 
fishing? What about the commute to work?), you go online to 
figure out what data is available that correlates reasonably well 
with those variables, you normalize the data to a 1-100 scale, 
pop it in a spreadsheet, and see what comes out. It is an impor- 
tant decision, to be sure, but not a very complicated one to think 
through logically. 


But even reasonably straightforward personal-level projects can get 
complicated quickly. Look at Rob's Should-|-Put-Solar-on-My-Roof 
logic tree and analysis. He had to calculate the potential value of 
the cost savings compared to the utility's rates and the investment 
required; he had to project whether the feed-in tariff (what the util- 
ity would pay him for excess power) would change and whether 
solar panels would continue to get cheaper; and he had to esti- 
mate his own carbon footprint to calculate whether this investment 
would substantially decrease his climate impact! As we saw back in 
Chapter 1, none of those analyses is beyond a smart homeowner, 
but Rob could have wasted a lot of time and effort if he didn’t plan 
these analyses with some thought. 


Maybe you think you would like to skip this chapter—it seems 
boring compared with the logical disaggregation trees of the last 
chapter, or the exciting use of big data techniques of Chapter 6. 
Don't do it! Having fun instead of drudgery in problem solving 
means finding the critical path. The critical path is the continu- 
ation of the prioritization process we described toward the end 
of Chapter 3. It means high-grading the analysis you use to solve 
problems so that you are always working on the parts that have 
the highest probability of yielding insight. Workplanning and 
the frequent iterations that follow the initial plan are the way you 
stay on that critical path. 


If you are working with a team, or can mimic that with input from 
family and friends, the workplan is the place to get specific about 
your initial hypotheses, clarify what outputs you want from analysis, 
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and assign the parts so that everyone knows what they are doing 
and by when. This is also the time to further establish and reinforce 
good team processes to avoid confirmation bias and other com- 
mon problem solving mistakes. 


This chapter covers best practices in work planning and pro- 
ject management, with examples of how you link your logic tree 
hypotheses to a plan for analysis. We discuss best practice in one- 
day answers, driving from your hypotheses. Finally, we introduce 
more team processes and norms to aid creativity and fight biases 
and cognitive errors in problem solving. 


Workplanning and Project Management 


Lots of people have experience with workplanning and project 
management. Let's face it, a lot of those experiences are prob- 
ably bad. Workplans are typically endless in length and full of 
quickly outmoded detail. Responsibilities and timing expecta- 
tions are vague and often violated, and unpleasant surprises in 
team follow-up meetings are all too common (“I thought you were 
going to interview John Snow” or “That isn't what | expected it to 
look like!”). 


For these reasons, we learned at McKinsey to do our workplanning 
with some twists (see Exhibit 4.1). 


These best-practice approaches to workplanning mean that 
we don't do any analysis for which we don’t have a hypothesis. 
We never go off and build a model without a very good idea about 
what question it answers. We sharpen our thinking even more by 
requiring that we can visualize what form the output might take 
(we call this dummying the chart), so we know if we would want 
it if we had it. We order our analyses logically, so those that can 
knock out, or obviate later analyses, are done first. And we are 
painfully explicit about who is expected to do what analysis by 
what date and time. By being specific here, we save tons of time 
in wasted work. 
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WHAT 


l 


e don't do any analyses that 
arent guided by very clear and 
testable hypotheses 


WHY 


There is no vague "II look into 
Kori 


2. We often work backward from Every analysis is driven by a 
clearly visualized and dummied’ specific contribution to the 
outputs solving of the problem 

3. We are very careful about the Do knock-out analyses first 
order in which we do analyses 

4. We are very specific about who o confusion about 
s doing what by when responsibilities or deadlines 

5. We have workplans that go out Early analysis always changes 
only 2-3 weeks, and longer the plan 
term study plans to rough out 
ater periods 

EXHIBIT 4.1 


Done well, it looks like Exhibit 4.2. 


We introduced the Bay Area Nursing example in Chapter 3. Let’s take 
a look at an example workplan from that bit of problem solving. You 
can see in Exhibit 4.3 the specificity at each level, issue, hypothesis, 
analysis, source of data, responsibility, and end products. 


There is a human inclination to just jump in to analysis without being 
crystal clear about what output is needed to solve the problem. 
Team members seem more prepared to spend time building models 
and conducting complex analyses than they are in carefully specify- 
ing what they really want from this work. With even more powerful 
analytic tools now available, including the learning algorithms of 
artificial intelligence techniques we'll talk about in Chapter 6, the 
urge to wade prematurely into data without doing the thinking on 
the underlying structure of the problem is even stronger than in the 
past. You should avoid this tendency. We encourage you to think 
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MODEL WORKPLAN 


» Starts with end 


» 


Responsibility 


91 


End Product 


» The design of 


Hypothesis Analysis Source ei 
up 4 *Timing 
The hypothesis » The analysis » The source » The team 
is a statement of a statement identifies the member and 


points (or leaves) 
from logic tree 

» The definition of an 
issue varies from ‘an 
important question 
to ‘an unresolved 


Definition 


question’ It is 
phrased so that it 
can be answered yes 
or no and on which 
a specific action 
depends. 


» Make sure each 
Issue Is stated in as 
detailed a manner 
as possible 

» Define sub-issues 
where necessary 


Action 


timing of delivery 
of end product 


likely locations, 
or means of 
obtaining data 
to undertake 
analysis 


of the ‘models 
that will be 
explored in 
order to prove 
or disprove the 


of the likely 
resolution of the 
issue. It includes 
the reason for 
answering yes 


or intermediate 
output 


the chart or 
table or other 
graphic that 
will show the 
relationship 
or lack of 
relationship 


» Draw ‘ghost’ 
exhibits 

» Develop story- 
line 


or no hypothesis, and 
hence resolve 
the Issue 
List all » Identify decision » Identify » Decide who will help 
hypotheses: use making readily collect the data 
» Front-line ideas » Determine available data and do analysis 
» Own ideas extent of » Decide on » Decide on time 
» Colleagues’ ideas analysis methodology frame, with 
» Discuss with required milestones 
team members » Simple case 
» Refine » Complex 
hypotheses justification 
» Readjust 
riorities for 
analysis 


EXHIBIT 4.2 


hard about problem structure and desired outputs before you start 


running numbers. 


As we discuss earlier, the order in which we do our analyses mat- 
ters. If you set up your workplan carefully, you'll do the knock-out 
analyses first, the really important analyses next, and the nice to 
have analyses last. In Rob’s solar case, the years to payback for 
his investment under a likely case power-pricing scenario was the 
knock-out analysis. If the payback was too long or uncertain, it just 
didn’t make sense to continue to the other analyses. Rob's question 
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WORKPLAN DETAIL: BAY AREA NURSING 


Issue Hypothesis Analysis Source Responsibility End Product 
Timing 
Why are fewer » Demand for » Assess number of » California » Liz (Monday » Longitudinal 
students nursing school qualified students Board of Jam - have a fun charts of nursing 
enrolling remains high, applying to nursing Registered weekendll) laces, applications, 
in nursing but program school, number Nurses » Ray (end of week). acceptance rates by 
school? capacity has enrolled, and number » American » Amory (Tuesday school 
declined turned away Association of next week) » Factor analysis 
» Inventory number and Colleges of of declines in 
size of nursing schools Nursing. enrollment. 
in the Bay Area » Interviews » Interview results 
» Identify reasons for with nursing 

declines in program school deans 

enrollment 
Why is the » Hospital RNs » Compare average » California » Margaret (end of » Longitudinal chart 
turnover rate are retiring age of hospital RNs Board of March) of retirement ages 
for hospital earlier than retiring to other RNs Registered » Paul (end of » Factor analysis of 
Registered expected andl and professions Nurses March) reasons. 
Nurses (RNs) or leaving » Assess primary » State-wide » Survey results 
highe hospitals for reasons for RNs nurse survey 


other sites or turning over 
professions 


EXHIBIT 4.3 


of next importance was whether it made sense to wait, due in this 
case to declining solar PV panel costs. In investment decisions, tim- 
ing is often critical even when the overall case for investment is 
strong. Rob and Paula are committed conservationists, but estimat- 
ing the reduction in carbon footprint, the third question, is really a 
nice-to-have in this case. On pure investment grounds this invest- 
ment has a 25% return, better than any interest rate out there. If the 
investment case was weaker, the carbon reduction might have 
pushed Rob to install the panels in any case, so it was important to 
include it in the tree—and it provides some emotional downside 
protection if some of the cost savings don’t materialize. 


Knock-out analysis involves making estimates of the importance 
of a variable and the influence you can have on it, an estimate of 
expected value, the first cut of which comes out of our prioritization 
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matrix from Chapter 3. As a result of a rough knock-out analysis, you 
typically conclude that some lines of enquiry aren't worth pursu- 
ing. For example, in reviewing business units in one of Rob's client's 
cost reduction exercise, a knock-out analysis concluded that a 50% 
gain from outsourcing was possible, which sounds great. But the 
contribution to that group to the overall company’s cost reduction 
target was only 5%. As a consequence no effort was focused on that 
particular unit, with the attention placed on units that would make 
a larger contribution to the target. This is consistent with the 80/20 
rule we will discuss in the next chapter: Focus your work on the 20% 
of the problem that yields 80% of the benefit. 


Chunky Workplans and Lean Project Plans 


Tools like Microsoft Project can be extremely helpful in workplan- 
ning. But they can also end up being like a beast that needs to 
be fed, generating enormous detail in workplanning that stretches 
out in time. As a new McKinsey engagement manager, before he 
developed the seven steps discipline, Charles was famous for work- 
ing all night to generate 30 page workplans for his teams. But with 
this much detail, they were at risk of being swallowed up by all the 
analyses that could be done, without any sense of critical path. 
These encyclopedic workplans were quickly outmoded when the 
first few analyses pointed to promising new avenues in the work. 
The way we do problem solving now—driving from hypotheses, 
using rough and ready analysis approaches wherever possible, fre- 
quent porpoising into the data, and constant iterations to refine 
the problem statement and logic tree understanding—we find that 
long workplans are unnecessary and unhelpful. 


Our approach is to do short, but highly specific, workplans that 
focus on the most important initial analyses, perhaps stretching 
out two to three weeks, and constantly revise them as new insights 
come from the team’s work. We couple these with rougher project 
plans, usually in Gantt chart format, that cover the fixed milestone 
dates and to ensure the overall project stays on track from a time 
perspective. We call them chunky workplans and lean project plans 
(see Exhibit 4.4). 
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USE CONCISE 2-3 PAGE WORKPLANS TO GUIDE PROBLEM SOLVING 2-4 WEEKS OUT 


Chunky Workplan 


Sub-Issue 


Hypothesis 


Analysis 


Source 


Responsibility/Timing 


What situation 
prevailed before 
the performance 
deterioration? 


» Refinery enjoyed 
niche market 
conditions, but 
had little growth 


» Assess transportation cost 
differential for imported 
products (net of crude 
disadvantage) 

» Compare PDQ crude cost 
advantage to alternative 

» Estimate tripoly impact 
(residual price differentials?) 


» Petroleum supply 
annual 

» Oilco World crude 
price data (Oil 
Institute Centre) 

» Residual analysis 

» Get Bluebeard price 
slate 


» Chris (Monday) 


» Chris (end of week) 


» Ned (Tuesday 
next week) 


What caused » Market niche » Conduct complete costs » Year-end products » Duncan (end of June) 


the decline in position did not assessment by component » John Doe 

performance? erode much, but 1988-1992 » Call Houston » Ned (Wednesday) 
operating costs, » Review CapEx plans experts 
SG&A and CapEx » Test both against industry » Soloman report » Chris (next Thursday) 
all grew faster best practice » PIRA 


than margins 


USE GANTT CHARTS TO MANAGE PROJECT DELIVERABLES OVER LONGER TIMEFRAMES 


STUDY PLAN 


Activity Z 
MAJOR REVIEW 
Activity X 


Activity Y 


Activity Z 


MAJOR REVIEW 


Avoid super detailed, encyclopedic “full project’ work plans 
No one reads them, they are out of date in 3 weeks, and no one wants to revise them. 


EXHIBIT 4.4 


One-Day Answers 


One-Day Answers 


We find there is great clarity in stating what you know about your 
problem at any point in the process. It helps bed down what under- 
standings are emerging, and what unknowns still stand between 
the answers and us. We call this a one-day answer, and they convey 
our current best analysis of the situation, complications or insight- 
ful observations, and our best guess at the solution, as we iterate 
between our evolving workplans and our analysis. By working this 
way we can divert resources to areas where we have the biggest 
gaps in problem solving and shut down analysis that is not taking 
us anywhere. We contrast this approach with typical efforts that 
encourage gathering huge data sets and endless interviews before 
problem solving starts. As analysis results come in, we can refine 
our one-day answers and begin to synthesize our evidence into 
more complete stories. We'll talk below about how we use other 
team processes to avoid the potential for confirmation bias that 
comes from early hypothesizing. 


There are many ways to structure one-day answers, but the classic 
way employed in McKinsey and elsewhere is to organize them in 
three parts: 


1. A short description of the situation that prevails at the out- 
set of problem solving. This is the state of affairs that sets up 
the problem. 


2. A set of observations or complications around the situation 
that creates the tension or dynamic that captures the problem. 
This is typically what changed, or what went wrong that created 
the problem. 


3. The best idea of the implication or resolution of the problem 
that you have right now. At the beginning this will be rough and 
speculative. Later it will be a more and more refined idea that 
answers the question, “What should we do?” 


We usually use this order of situation—-observation-resolution in 
our team processes, but later on as you begin to synthesize your 
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findings, you may reverse these and lead with “You should do this,” 
then followed by supporting reasons. Chapter 7 explains this in 
more detail. 


One-day answers should be crisp and concise. This is not the place 
to have 40 bullet points of unorganized facts. Exhibit 4.5 gives some 
best-practice thoughts. 


WHAT ONE-DAY ANSWERS ARE NOT 


NOT THIS 


Situation 


Observation 


» The problem or opportunity at 
the core of the decision-maker's 
dilemma 


» The critical insight or leverage 
point that is emerging 


» The logical high ground 


» Issue history or masses of 
facts unrelated to the core 
opportunity/problem 


» Vague description of the 
complication (rehash of problem 
statement) or another bunch of 
facts masquerading as insight 


» The one and only answer 


Implication »Implied set of actions » Unsupported prejudices or 
» Options we see at this point preconceived ideas not related to 
steps | and 2 
EXHIBIT 4.5 


Let's look at a short example from our hardware company case study. 


Situation: Hechinger is a dominant player with a long and suc- 
cessful history in one region and seeks to expand. 


Complication: A new competitor, Home Depot, has emerged 
with a warehouse superstore model that appears to be growing 
much faster, driven by substantially lower pricing. Their business 
model is different, and compensates for lower prices by sourcing 
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economies, lower cost logistics, and higher asset productivity. 
They will soon have geographic overlap with Hechinger. 


Resolution: To remain competitive, Hechinger needs to quickly 
reform its inventory management and logistics systems, and 
develop lower-cost sourcing models, to allow lower pricing—or 
it will face a real threat even in its core markets. 


As you can see, this is a concise summary of where the problem 
solving effort sits at a moment in time. It guides the analysis that 
still has to be done (in this case particularly the answer to the ques- 
tion “What should Hechinger do?”). It provides a strawman that 
can (and should) be attacked and stress tested for faulty think- 
ing. And it gives you something to say if you find yourself in an 
elevator-test moment! 


Great Team Processes for Workplanning 
and Analysis 


Nearly everyone has experience working in teams, and we're will- 
ing to bet you have worked on good ones and bad ones. Our task 
is focused on problem solving, not good management practices 
more broadly, but there is lots of overlap. We have found that 
the best teams have relatively flat structures, good processes and 
norms, and explicit approaches to avoiding bias. Let's take each of 
these in turn. 


Team Structure and Leadership 


Good problem solving teams usually have an excellent lead or 
coordinator—somewhere between a musical conductor and an air 
traffic controller—who makes sure that the basic elements come 
together and on time. But the best teams typically have relatively 
little hierarchy in the structure of brainstorming and ideation. This 
kind of structure can be challenging to create inside larger organi- 
zations that have conventional hierarchical roles. When we work 
within large organizations, we nearly always set up temporary teams 
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that act outside the normal reporting structures and deliberately 
have limited lives. This allows for non-hierarchical and creative team 
processes that are more likely to generate good solutions. 


Of course, these same principles apply to working on family-level 
problems, and to social-scale problems with decision makers inside 
political hierarchies. In every case we try to make conventional 
authority figures play a different and less directive role in problem 
solving to get fresh and creative answers. 


We will deal with some important insights on team composition 
next. 


Team Norms 


Over the course of our careers, we have seen some teams that prob- 
lem solve in a highly efficient way and others that flail around, miss 
deadlines, and fail to develop compelling solutions. This can hap- 
pen even after good work on problem definition, disaggregation, 
and prioritization. What is it that good problem solving teams do 
better in their approach to workplanning and analysis? A number of 
things stand out in our experience: 


1. They are hypothesis driven and end-product oriented. They use 
strong hypotheses to guide workplanning and analyses. This 
doesn’t mean they are trying to prove the hypothesis in spite 
of the facts. Just the opposite: Strong hypotheses are easier to 
challenge and pressure test. The end products we are describ- 
ing are typically analyses that will unambiguously resolve an 
issue or sub-issue. For Rob’s knee arthroscopy, discussed in the 
next chapter, he framed the hypothesis that he should have 
surgery now and then set out the supporting analyses. He set 
a clear threshold for those: that the outcome would be bet- 
ter than doing nothing and that no improved technology solu- 
tion was imminent. The end product of his analysis was the 
completed decision tree where he compiled research studies 
and the latest timing of new treatments such as stem cells for 
meniscal repair. 
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2. They porpoise frequently between the hypothesis and data. 
They are flexible in the face of new data. This means they are 
ready to cease analysis where it has reached a dead end and 
start work on promising new lines of inquiry. In the Avahan 
HIV project in India that we introduced in Chapter 2, the team 
started with an initial hypothesis that was directionally useful 
but provided no leverage for action. What did they do? They 
took a new tack based on what they had learned that focused 
on female sex workers and the communities in which they lived. 
This hadn't previously been explored as part of a disease prob- 
lem, and it ultimately yielded insight and a course of action of 
profound impact. 


3. They look for breakthrough thinking rather than incremental 
improvements. They do this by asking about performance limits 
and reference points for best practice, and the distance they are 
from those limits. A mining company using large, heavy vehicles 
was exploring whether they could do tire changes more quickly. 
They asked who did tire changes fastest and had the most expe- 
rience in the world. The answer was Formula 1. The mining team 
then visited Formula 1 racing pits to learn about how they could 
improve their processes. That is creativity! 


Team Behaviors to Avoid Bias and Error 


Many of you will be familiar with the notion of type 1 and type 2 
thinking patterns, introduced to the general public in Nobel Prize 
winner Daniel Kahneman’s Thinking, Fast and Slow.' Type 1 think- 
ing is automatic and intuitive, and governs much of our behavior. 
Perhaps to save us from being eaten by jackals, many of our deci- 
sions are essentially automatic, without the conscious logic of type 
2 considered, rational thinking. Any time we have done something 
before and had it be successful, it is more likely that our brains will 
quickly choose this path again, without conscious thought. Just 
like learning to say a foreign phrase, the first time is labored and 


‘Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and 
Giroux, 2011). 
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painfully self-conscious. With time it becomes, literally, second 
nature and disappears into our automatic mental systems. 


This is great for speaking a new language or riding a bike or tying 
your shoes . . . but not so great for systematic problem solving. Our 
kind of problem solving can and does benefit from pattern recog- 
nition, of course. When you correctly see that a particular business 
problem is best cleaved by a return on capital tree or a supply curve, 
it makes problem solving faster and more accurate. Philip Tetlock 
reports that grandmasters at chess have 50,000-100,000 patterns 
stored in this deep memory system.? But when we incorrectly see 
a familiar framing for a new kind of problem, we risk disastrously 
wrong solutions or endless work getting back on track. This kind 
of mistake is sometimes called an availability heuristic (you use 
the framework you happen to have handy, not the right one) or 
substitution bias (you substitute a simple model you know rather 
than understanding the more complicated actual model). And even 
when there is no absolute cognitive mistake in problem framing, 
excessive problem patterning is a block on novel solutions and 
creativity in problem solving. 


Now here is the bad news. Depending on who is counting,’ there 
are upwards of one hundred different kinds of cognitive biases 
and mistakes that humans are prone to make in problem solving, 
some related to type 1 fast thinking errors, some driven by other 
processing biases, including many related to our relationship 
with resources. The enumeration of cognitive biases in decision 
making coming out of experimental psychology and behavioral 
economics is both interesting and important for anyone who cares 
about problem solving. But to believe some of the literature, these 
biases render people incapable of rational thought entirely! In our 
experience good problem solving design and team processes can 
address most of these, and in addition encourage creativity. This is 
what the seven steps process is all about. 


Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of 
Prediction (Crown Publishing, 2015). 
3See Rolf Dobeli, The Art of Thinking Clearly (Sceptre, 2013). 
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A Practical Way to Deal with Biases 


Lets talk about the main sources of bias and error in problem solv- 
ing. In conversations with our former colleague and Kahneman 
collaborator, Professor Dan Lovallo, he suggests that the most 
important to address are confirmation bias, anchoring bias, and 
loss aversion.* We will add in availability bias and overoptimism 
as additional issues to address in your team processes. Indeed, 
as Exhibit 4.6 shows, many of the other biases described in the 
literature are forms of these five underlying issues. 


1. Confirmation bias is falling in love with your one-day answer. 
It is the failure to seriously consider the antithesis to your the- 
sis, ignoring dissenting views—essentially picking low hanging 
mental fruit. 


2. Anchoring bias is the mistaken mental attachment to an initial 
data range or data pattern that colors your subsequent under- 
standing of the problem. 


3. Loss aversion, and its relatives, the sunk cost fallacy, book loss fear, 
and the endowment effect, are a failure to ignore costs already 
spent (sunk) or any asymmetric valuing of losses and gains. 


4. Availability bias is use of an existing mental map because it is 
readily at hand, rather than developing a new model for a new 
problem, or just being influenced by more recent facts or events. 


5. Overoptimism comes in several forms including overconfi- 
dence, illusion of control or simply failure to contemplate 
disaster outcomes. 


There are a number of team process-design approaches that can 
lessen the impact of most of these cognitive biases and errors, and 
that can simultaneously encourage more novelty and creativity in 
problem solving. 


‘Daniel Kahneman, Dan Lovallo, and Olivier Sibony, “The Big Idea: Before You 
Make the Big Decision,” Harvard Business Review, June 2011; Private conversation 
with Professor Dan Lovallo, University of Sydney. 
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DEALING WITH BIASES 


Principal Bias 


CONFIRMATION 
BIAS 


ANCHORING 
BIAS 


LOSS 
AVERSION 
BIAS 


AVAILABILITY 
BIAS 


OVEROPTIMISM 
BIAS 


EXHIBIT 4.6 


Related Bias 


» Affect heuristic 

» Group think 

» Self-interest bias 

» Authority bias or 
Sunflower bias 


» Saliency bias 

»Neglect of 
reversion to mean/ 
compounding 


» Sunk cost fallacy 

»Endowment effect 

» Accounting book 
loss fear 


» Substitution bias 
» Overemphasis given 
to current events 


» Overconfidence 
»lllusion of control 
» Disaster neglect 
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What is it? 


Falling in love 

with your one-day 
answer: failing to 
look at alternatives 
and antitheses; not 
hearing dissenting 
views 


Assuming an initial 
data range or 
increased/decreased 
pattern Is the 
whole range or will 
continue (mental 
numerical stickiness) 


Mental numerical 
stickiness or any 
kind of asymmetric 
valuing of losses or 
gains 


Use the map or story 
we have at hand 

rather than investing 
in understanding the 
new complex situation 


Underestimating low 
probability events 


How to deal? 


» Constructive 
confrontation 

» Dialectical argument: 

thesis, antithesis, 

synthesis 

» Team composition— 

diversity (views/ 

background) 

»Role playing 


Develop alternative 
hypotheses (anti- 
theses) 

Hard-core data 
gathering and analysis, 
including scenario 
modeling 

»What you have to 
believe—team process 


» Clear forward-looking 
mind-set 

» Good analytic tools 
(NPV analysis valuing 
options, Bayesian 
thinking) 


»Value of additional 
information 

» Explicit options to 
wait vs. decide now 


» Explicit extreme 
downside case 
» Premortems 


Diversity in team members: Building a team with different back- 
grounds and viewpoints really helps create an environment of 
active openness to new ideas and approaches. This is a great 
source of creativity in problem solving and often overlooked. If you 
are solving your problem solo, find a way to brainstorm your ideas 
with a diverse range of others, insiders and outsiders, at each 
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stage of the process. Phillip Tetlock’s work on forecasting shows 
that teams always outperform individuals in this form of prob- 
lem solving, even very high-performing solo practitioners. And 
the best superforecasting teams even beat crowd-sourcing and 
prediction markets.° 


Always try multiple trees/cleaves: Even when a problem seems per- 
fectly designed for your favorite framework, try multiple cleaving 
frames to see what different questions and insights emerge. When 
we decided to look at obesity as a wicked problem, discussed in 
Chapter 9, we had a team meeting where we tried out various 
alternative-cleaving frames. In only half an hour we outlined a num- 
ber of possible cleaving frames and argued the case for each—such 
as incent/regulate, individual behavior/public health, incidence/ 
severity and supply/demand, before settling on supply/demand. 
Don't settle on your first cut through the problem. 


Try adding question marks to your hypotheses. We are committed to 
being hypothesis driven, but our former colleague Caroline Webb's 
work reports behavioral research suggesting that active question- 
ing moves our brains into discovery mode, away from confirmation 
mode.ć A number of the best problem solvers we've met use active 
questioning as their prime problem solving tool. The answers they 
get lead to the next set of questions until finally the hypothesis is 
sharpened or challenged in its present form. 


Team Brainstorming Practices: The values your team establishes 
early on contribute enormously to the quality of your problem solv- 
ing. Here are some practices we have found especially conducive 
to fighting biases and promoting creativity: 


e Obligation to dissent 
A core value of McKinsey is the obligation to dissent and it is 
baked into even the greenest business analysts from induction. 
This means not only is it okay for a junior team member to speak 


Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of 
Prediction (Crown Publishing, 2015). 
6Caroline Webb, How to Have a Good Day (Random House, 2016), 167. 
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up when they disagree in a brainstorming session—or in front 
of the client—it is an absolute obligation. Senior team mem- 
bers who don't adhere to this value can be fired. Team members 
typically respect hierarchy and need team leaders to encourage 
them to speak up if they have a different view, whether they have 
been employed for a day or ten years. 


Role playing 
Try acting out your interim solutions from the perspective of cli- 
ents, suppliers, other family members, citizens . . . whoever isn't 


you. Then switch roles and play a new character. It sounds embar- 
rassing—and it is at first—but it is remarkably clarifying. Caroline 
Webb also recommends drawing, mapping, or verbalizing solu- 
tions as alternatives to conventional discourse to generate differ- 
ent and more creative results.’ 


Dialectic standard 

Establish the classical argument form as your team norm: thesis, 
antithesis, synthesis. This means every idea or hypothesis must 
be met with its antithesis and challenged, before joining the 
learning together in synthesis. 


Perspective taking 

Perspective taking is the act of modeling another team mem- 
ber’s assertion or belief (especially if you don’t agree) to the point 
that you can describe it as compellingly as the other.® 


Constructive confrontation 

To disagree without being disagreeable is the heart of great prob- 
lem solving team process. One of the great tools we both used 
in McKinsey is “What would you have to believe?” to accept a 
particular thesis or viewpoint. This involves spelling out all of the 
assumptions implicit in the perspective, and all the implications. 
One example of “What would you have to believe?” involved 
looking at a growth company’s share price during the dot-com 
era. To justify the share price earnings had to grow at 40% com- 
pound for the next five years, then at double the US GDP growth 


7Caroline Webb, How to Have a Good Day (Random House, 2016), 170-172. 
8Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of 
Prediction (Crown Publishing, 2015). 
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rate in perpetuity! That is an unbelievably high growth expecta- 
tion... and was not achieved, leading to a stock price crash for 
the company. 


e Team distributed voting 

There is often disagreement when pruning branches on a logic 
tree, or when assigning priorities in the analysis work planning. 
If you aren't careful, the most articulate or senior team mem- 
ber can inappropriately sway the conversation. As described 
in Chapter 3, one approach we have used is to assign each 
team member 10 votes, represented by sticky notes, and have 
each team member use them to vote on their favorite (or least 
favorite) analysis, allowing cumulative or bullet voting—most 
senior person votes last, so as not to bias the choices of more 
junior members. 


e Solicit outside views (but be careful with experts) 

It is easy to breathe your own exhaust in intensive problem solv- 
ing, especially when you are working in a space or business that 
is familiar. It is critical to engage outside data and perspectives 
to refresh your view of the problem. The normal thing is to inter- 
view experts—but the risk is that they just reinforce the dominant 
or mainstream view and therefore smother creativity. Try talking 
to customers, suppliers, or better yet players in a different but 
related industry or space. 


Explicit Downside Scenario Modeling and Pre-Mortem Analysis: 
Nicolas Taleb’s wonderful book The Black Swan’ reminds us of 
the dangers of assuming normal distributions. Improbable jumps 
or discontinuous events do occur, and long-tailed distributions 
(irregular-shaped statistical distributions with a long tail on one 
side, indicating small but measurable probability for even extreme 
events) mean we should take these into account in our problem 
solving and modeling. This means that we should explicitly model 
not just a narrow band (expected case, downside case) of values, but 
more extreme downside cases. A related idea is to do pre-mortem 


°Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable 
(Random House, 2007). 
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analysis, where the team works through all the implications of even 
unlikely failure cases.'° 


Good analytic techniques: Humans are prone to a number of errors 
around money. These include paying attention to sunk costs, unrea- 
sonably high discount rates on future events, and treating losses and 
gains asymmetrically. One way to avoid these errors is to use good 
analytic techniques, including problem/model design, net present 
value analysis, marginal analysis, and use of cash flows rather than 
accounting book values. We will cover these in Chapter 5. 


Broaden your data sources: In every area of life, individual/ 
workplace/society, there are core government and private data 
sets that everyone has access to. Sometimes these are terrific, 
but everyone has them, including your competitors, and there are 
often methodological issues in their collection. It is worth consider- 
ing whether there are options for crowd sourcing alternative data, 
whether prediction markets cover your topic or could be induced to, 
or whether your interest area is amenable to some form of A|B test- 
ing or randomized controlled trials. Custom data collection costs 
have come down substantially and new data sets can yield insights 
very different from the obvious mainstream analyses. Tools such as 
Survey Monkey are simple to use and can bring customer, competi- 
tor, and supplier perspectives that may be especially insightful. 


How does this all get put in place in real-life settings in your teams? 
Often by the cues and behaviors of the team leader, like the case 
in Exhibit 4.7. 


Agile Team Processes 


In the Introduction we argued that many organizations are respond- 
ing to demands to be become better problem solvers by taking 
on the persona of problem solving corporations. They do so by 
seeking to become lean and agile and often employ the language 


Daniel Kahnemann, Dan Lovallo, and Olivier Sibony, “Before You Make That Big 
Decision,” Harvard Business Review, June 2011. 
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TACKLING BIASES IN MINERAL EXPLORATION 


Rob saw a powerful way to address major biases in a McKinsey team setting some years ago in a 
project led by Bob Waterman, co-author of the best-selling book /n Search of Excellence. The team 
was reviewing the exploration strategy of an Australian mineral resource company that hadn't 
made a major discovery in a decade, despite significant spending on exploration. Prior to this they 
had made numerous world-class discoveries 


Loss Aversion versus Big Bets 

The team reviewed and gathered results for the client's failing exploration strategy (selection of 
prospects, Funding allocations, sampling and drilling). Nothing definitive was emerging to explain 
what was happening. In interviews with the management team, however, Bob detected a difference 
between the stated goal—to locate scarce resources that could become long-lived mines in the 
bottom quartile of operating cost (which requires a big bet mentality)—and the way they were 
approaching exploration. They had shifted away from making big bets because of a fear of losses 
(loss aversion), to making small bets where there was a low probability of loss (typically around 
known mineralization where technical successes were more likely). Bob then proposed the team 
look outside the group where success and failure could be contrasted 


Anchoring Outside versus Inside 

Companies that anchor outside their own teams and take a scientific view of the odds rather 
than limit themselves to internal data or the case at hand, unlock a different perspective. With 
the client's help, the team surveyed a dozen mineral exploration companies, half of whom had 
been very successful and the other half unsuccessful. While a small sample, it represented more 
than 75% of Australian mineral exploration in a year. The client agreed to share its findings 
with all participants. What became clear from the survey was that the successful companies had 
big bet cultures and were science led, quickly testing their concepts through drilling. Based on 
those drilling results they either doubled down or abandoned the mineral prospect. Unsuccessful 
companies were less hypothesis-driven, sticking with unsuccessful prospects for longer. Anchoring 
outside became a new best practice in exploration 


Authority Bias 

Bob presented the team’s findings at a meeting with the CEO, senior management, and the 
exploration team. As soon as Bob finished the presentation, the CEO began speaking. Bob thought 
this would be the case and was worried it might stifle dissenting views. He intervened, suggesting 
instead that each person at the table give their views first and that the CEO then make concluding 
remarks and summarize the discussion. As they went around the table, each person voiced their 
support for a new finder mentality and rigorous science-based processes. They were able to 
speak their minds without fear of opposing the CEO. This simple technique reduced the prospect 
of authority bias, a subset of confirmation bias. The client successfully implemented changes to 
elevate the importance of exploration and their science capability. Not long after, they ended 
their drought with exploration success 


EXHIBIT 4.7 


of scrums, a term from rugby. They seek to work on the right 
problems, address root causes of problems, and organize around 
teams with short duration work plans and clear accountability (see 
Exhibit 4.8). All this is consistent with the approach we have outlined 
in this chapter. 
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AGILE ORGANIZATIONS, SCRUMS AND SPRINTS 


oppure ter 


The language associated with teams is changing to embrace agile, lean, and scrums. We have daily 
scrum events, scrum masters, sprint planning. sprint reviews, and sprint retrospectives—all within 
a month. Teams and organizations who have had success in applying the principles point to a focus 
on priorities, timely outcomes, accountability, transparency, and a bias for action. As the metabolic 
rate of organizations quickens the need for agile organizations and scrum teams can be expected 
to rise further. 


There is commonality and difference between what is termed lean, agile and scrum. Lean has a focus 
on decreasing waste as did its predecessor, kanban. Agile has a focus on increasing velocity. Scrum 
is described as a framework for developing complex products. All three use team structures to bring 
openness and accountability to tasks through project planning descriptors 


Some see the rise of agile and scrums as heralding a new organization paradigm and providing 
a framework to address complex adaptive problems ( Dr. Jeff Sutherland, 1995). To meet that 
expectation organizations will need to become problem solving corporations as we discussed in the 
introduction, displaying mental muscle and machine muscle. They will need to embrace bulletproof 
problem solving as a requirement, At the team level, agile and scrum teams have to ask themselves 
the following questions if their problem solving is to be as good as it can be 


» Are we working on the right probleme (Step 1) 
» Have we broken down the problem into key issues to analyse? (Step 2) 
» Are our priorities right in terms of short and long-term impact? (Step 3) 
» Have we balanced the scrum deliverables against burn-out risk of team members? (Step U) 
» Have we brought outside perspectives and expertise to bear? (Step 4) 
» Are complex problems that require analysis getting the right share of resources? (Step 5) 
» Are we carefully synthesizing our findings? (Step 6) 
» Are we presenting our findings in a compelling narrativee (Step 7) 
» Is there adequate iteration between progress and goals? (Steps 1-7, again) 


We see great potential for organizations to build on the tenets of agile, lean and scrums to 
become problem-solving corporations. To do so it means that the key elements of the way teams 
do bulletproof problem solving that we have outlined need to be Inculcated into the organization 


EXHIBIT 4.8 


Chapter 4 Takeaways 


PSYCHOLOGICAL PROFILE OF SUPERFORECASTERS 


Philosophy Style Methods 
Cautious Actively open-minded Pragmatic 
Humble Intelligent & well-read Analytical 
Non-deterministic Reflective Dragonfly-eyed (multiple lenses) 
Growth mindset Introspective Probabilistic 
Lots of grit Self-critical Thoughtul updaters 
Numerate 
EXHIBIT 4.9 


Above all, there is an orientation or attitude that is found in the 
best problem solvers that reflects an active openness to new 
ideas and data, and a suspicion of standard or conventional 
answers (see Exhibit 4.9). Tetlock describes it well in profiling the 
superforecasters. 


In closing, good workplans make an enormous difference in prob- 
lem solving. We have explained how they are a necessary but not 
sufficient condition for problem solving success. One-day answers 
help sharpen our hypotheses and make our analysis focused and 
efficient. Knock-out analyses provide focus on the critical path in 
problem solving. Finally, there are a host of team effectiveness dis- 
ciplines that accompany the workplan that can help you produce 
great outcomes and guard against the pitfalls and biases that we 
hear so much about. 


Chapter 4 Takeaways 


e Good discipline and specificity in workplanning will save you a 
huge amount of work—take the time to do it right up front! 

e Tobe onthe critical path in problem solving, make sure you get the 
order of your analyses right and do the knock-out analyses first. 
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Build a Great Workplan and Team Processes 


Make your workplans chunky: short and specific; and your study 
plans lean: capture key milestones so you deliver on time. 


One-day answers clarify where you are and what work is left to 
do—and provide a strawman for pressure-testing your work. 


Tools like pre-mortems can help highlight risks to focus on with 
big decisions where a lot is at stake. 


Good team structure and norms help foster creativity and fight 
bias in problem solving. 


Short-circuit hierarchy wherever you can—foster an obligation 
to dissent. 


Try role-playing for creativity to bring in outside perspectives. 
Great problem solvers are well read, open to new ideas, reflective, 
self-critical, persistent . . . and use teamwork wherever they can 
Agile team processes are increasingly the vehicle for problem 
solving efforts; try to build agile characteristics into your teams. 


Problems to Try on Your Own 


íl; 


For the problem “Should | buy an electric car?”: Design a 
logic tree of questions in the best knock-out order; show how 
it is MECE. 


. Write up the one-day answer for the electric car decision. 


. Try role-playing support for Brexit; then switch sides and make 


as compelling a case for Remain. 


. Re-read the nursing case study and make the conclusions in a 


deductive logic form then an inductive logic form. 


. List out the most common biases that affect problem solving 


and how you plan to address them in your problem solving. 


. Take whatever your current problem is and dummy up 10 exhib- 


its you would like to have real data and analysis for. 


. Prepare a one page, agile plan for the next month for a project 


you're working on that incorporates the seven-steps process. 


Chapter Five 


Conduct Analyses 


x 
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How you go about gathering facts and conducting analysis to 
test your hypotheses often makes the difference between good 
and bad problem solving, even when the earlier steps have been 
followed carefully. Good problem solvers have a toolkit at their 
disposal that helps them work efficiently, starting with heuristics 
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and rules of thumb to understand the direction and magnitudes 
of relationships that allows them to focus attention on the most 
important issues. They don't jump right into building giant models 
until they have a clear understanding of whether and where complex 
tools are required. 


The analysis stage is critical to the objectivity of your problem 
solving. There are many jokes about torturing the facts until they 
speak the truth you want to hear. There was even an old New Yorker 
cartoon that showed a person reaching into a filing cabinet with 
drawers marked “His facts. Her facts. Your facts.”—and this was 
well before the advent of fake news. Following a structured prob- 
lem approach, where we use good analytic techniques to pressure 
test our hypotheses and good team processes to limit bias, allows 
us to avoid torturing the facts. 


In this chapter, we focus on how to do powerful analyses quickly and 
efficiently, starting with heuristics, shortcuts, and rules of thumb. We 
illustrate how you can structure and resolve many analytic issues 
you face with straightforward heuristics well before any use of the 
complex “big guns” analysis we introduce in Chapter 6. 


Heuristics and Rules of Thumb 


Heuristics are powerful tools that act as shortcuts in analysis. They 
help you size the different elements of the problem to determine 
the efficient path in further analysis. They can be dangerous when 
incorrectly applied, of course. Author Nicholas Taleb describes how 
simple rules based on past success can lead you astray in settings 
where low probability events can produce outsized errors.' We'll 
show you how to avoid these mistakes. 


We use the term heuristics, rules of thumb, and shortcuts inter- 
changeably. In Exhibit 5.1, we explore a range of heuristics that are 
useful for individuals, enterprises, and citizens. When you are asked 
to list your top 10 songs of all time, it’s hard to leave some classics 


‘Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable 
(Penguin, 2007). 


HEURISTICS 


& SHOKT CUTS 


What it is 


Heuristics and Rules of Thumb 


When to use it 


Watch out 


Occam's Razor 


Order of 
Magnitude cuts: 
bounding the 
market 


80/20 Thinking 


Rule of 72 


S-Curve | 
Adoption Curve 


Expected Value 


Bayesian Thinking 


Reasoning by 
Analogy 


Break-even Point 


Marginal Analysis 


Distribution of 
outcomes 


EXHIBIT 5.1 


» The simplest solution 
that requires the 
fewest assumptions 


» What is maximum 
potential value? 


y 


Finding the 20% of the 
roblem that drives 80% 
of the value 

Finding the big levers 


¥ 


y 


Divide growth rate into 
72 to get doubling 
eriod 


» Typical model of adoption 


» Expected value is the 
value of an outcome 
multiplied by its 
probability of occurring 


» Conditional probability 


» Creating reference 
classes 


» Break-even volume 


» Economics of the 
next unit 


» The possible range of 
outcomes 


» Always 


» Always! Never start a 
problem without order 
of mag analysis 


» Scoping problems 
under constraint 


» Any growth or 
compounding problem 


» Adoption of new 
technologies and 
products 


» Any time you have 
a future uncertain 
event involving value 


» When you need to 
think probabilistically 


» When order of magnitude 
varies widely depending 
on comparator 


» Quick check on 
business model 
viability 


» Production, consumption, 
and investment problems 


» Project cost estimates 
» New business revenue 


» M&A 


» Don't get 
committed to the 
first cut answer 


» Don't fall in love 
with the maximum 


» Don't miscalculate 
risks in large scale, 
interconnected 
settings 


» Step change 
process 


» Slow then rapid 
diffusion 


» Other/better 
disruptive 
technologies 


» Care to assemble 
data for 
calculations 


» Outlier cases 


» Fixed costs with 
scale behavior 


» Step jumps in cost 


» Mean reversion 
» Non-normal 
distributions 
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off the list. So too it is with our top heuristics. We didn’t invent them, 
we just make good use of them and so can you. 


The oldest of these is definitely Occam's Razor—favor the simplest 
solution that fits the facts—which originated in the fourteenth cen- 
tury. It tells us to select the hypothesis that has the fewest assump- 
tions. One way of seeing why this make sense is a simple math 
example: If you have four assumptions that are independent of 
each other, with an 80% separate chance of being correct, the prob- 
ability that all four will be correct is just over 40%. With two assump- 
tions and the same probabilities, it is 64%. For many problems the 
fewer the assumptions you have the better. Practically speaking, 
this means avoiding complex, indirect, or inferential explanations, 
at least as our starting point. Related to Occam's Razor are one- 
reason decision heuristics, including reasoning by elimination and 
a fortiori reasoning, where you eliminate alternatives that are less 
attractive.2 The important reminder is not to get committed to a 
simple answer with few assumptions when the facts and evidence 
are pointing to a more nuanced or complex answer (remember the 
availability and substitution biases from Chapter 4). 


Order of magnitude analysis is used to prioritize team efforts by esti- 
mating the size of different levers. In business problems, we typically 
calculate the value of a 10% improvement in price, cost, or volume 
to determine which is more important to focus on (assuming, of 
course, that each is similarly difficult or easy to change). It applies 
to analyzing social issues as well. The nursing case in Chapter 3 
showed the difference in outcomes from improvements in nurse 
quality and quantity. Doing an order of magnitude analysis should 
provide a minimum, most likely, and maximum estimate, not simply 
the maximum. Often the best knock-out analysis, which we referred 
to in the previous chapter, is an order of magnitude analysis to show 
how changes would improve an outcome or make little difference. 


Efficient analysis is often helped by the 80:20 rule, sometimes called 
the Pareto Principle after the Italian economist Vilfredo Pareto, who 


?Gerd Gigerenzer, Peter M. Todd, and the ABC Research Group, Simple Heuristics 
That Make Us Smart (Oxford University Press, 2000). 
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first noticed this relationship. It describes the common phenom- 
enon that 80% of outcomes come from 20% of causes. If you plot 
percent of consumption of a product on the Y-axis and percent of 
consumers on the X-axis you will often see that 20% of consumers 
account for 80% of sales of a product or service. The point of doing 
80:20 analyses is again to focus your analytical effort on the most 
important factors. Many business and social environments feature 
market structures where 80:20 or something close to that ratio is 
the norm, so it’s a handy device to use. For example, in health- 
care 20% of the population often account for 80% of healthcare 
costs, a fact that provides huge difficulty for insurance plan-rating 
models. The 80:20 rule may also apply in complex system settings. 
For example, when Rob visited Myanmar in 2016 he learned that 
only one-third of the population have access to electricity, yet the 
country has the potential to produce 100 GW from hydropower. 
Ateam explored the trade-offs between fishery support and hydro- 
power by looking at portfolios of projects on a tributary of the great 
Irrawaddy River. They showed you could get almost 95% of poten- 
tial power for a 20% reduction in fishery supports. To achieve 100% 
of power entailed losses of one half to two-thirds of fishery sup- 
port. This extreme version of the 80:20 rule starkly highlights the 
dilemma for policymakers, in this case between energy, livelihoods, 
and the environment. 


Compound growth is key to understanding how wealth builds, how 
enterprises scale quickly, and how some populations grow. Warren 
Buffett said: “My wealth has come from a combination of living 
in America, some lucky genes, and compound interest.”* A really 
quick way to estimate compounding effects is to use the Rule 
of 72.° The rule of 72 allows you to estimate how long it takes for 
an amount to double given its growth rate by dividing 72 by the 


3Report prepared for the United Kingdom's Department of International Devel- 
opment by The Nature Conservancy, WWF, and the University of Manchester, 
“Improving Hydropower Outcomes through System Scale Planning, An Example 
from Myanmar,” 2016. 

‘Warren Buffett, “My Philanthropic Pledge,” Fortune, June 16, 2010. 

5Our friend Barry Nalebuff of Yale points out that the actual rule is 69.3, but is 
usually rounded up to 72 because it is easier to do the division in your head. 
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rate of growth. So, if the growth rate is 5% an amount will double 
in about 14 years (72/5 = 14.4 years). If the growth rate is 15%, dou- 
bling occurs in four to five years. In a team meeting, Rob asked our 
research team what a $1000 investment in Amazon would be if you 
invested at the time of the initial stock offering in 1997. Charles 
thought about it for about 90 seconds: He tried a low rate of com- 
pounding of 5%, where doubling occurs every 14 years, then a high 
rate of 50% where doubling occurs every 18 months, before settling 
on $100k. The actual answer is $83k based on a 36% compound- 
ing rate, doubling every 2 years. Pretty good with no facts, just the 
Rule of 72! Where do errors occur with the rule of 72? When there 
is a change in the growth rate, which of course is often the case 
over longer periods. This makes sense, as few things continue to 
compound forever (try the old trick of putting a grain of rice on 
the first square of a chessboard and double the number on each 
successive square). 


A useful heuristic if you are involved in estimating the adoption rate 
for a new innovation is the S-curve, which shows a common pattern 
of sales with a new product or a new market. S-curves are drawn 
with the percent of full adoption potential on the Y-axis and the 
years since adoption on the X-axis. The shape of the S will vary a lot 
depending on the reference class you select and particular reasons 
for faster or slower take up in your case. Charles led a successful 
start-up in the early days of Internet adoption. At that time, many 
forecasters overestimated the impact of Internet penetration in 
the short term (think of Webvan or Pets.com), but underestimated 
adoption in the longer term of 10 to 15 years. It looks in hind- 
sight very clearly to be a classic S-curve. In 1995 fewer than 10% 
of Americans had access to the Internet. By 2014 it reached 87%. 
The S-curve can take many specific profiles as Exhibit 5.2 shows. 
Like any heuristic, you don’t want to apply this rigidly or blindly, but 
rather use it as a frame for scoping a problem. The challenge is to 
get behind sweeping statements like “the world has speeded up,” 
to understand why a particular technology will have adoption at a 
certain rate. 


Heuristics and Rules of Thumb 


HISTORIC ADOPTION CURVES FOR TECHNOLOGICAL INNOVATIONS 
ADOPTION TREND BY TECHNOLOGY 


90% adoption 


50% adoption 


% of full adoption potential 


Years since adoption 


Fast case 


Slow case 


EXHIBIT 5.2 


Source: McKinsey Global Institute, A Future that Works: Automation, 
Employment, and Productivity, McKinsey Global Institute, January 2017. 


Expected value is simply the value of an outcome multiplied by 
its probability of occurring. That is called a single point expected 
value, but it is usually more useful (depending on the shape of 
the distribution) to take the sum of all probabilities of possible 
outcomes multiplied by their values. Expected value is a power- 
ful first-cut analytic tool to set priorities and reach conclusions on 
whether to take a bet in an uncertain environment. As an exam- 
ple, in venture capital the goal could be expressed as the desire 
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to create a unicorn—a billion-dollar company. The probability of 
reaching unicorn status calculated for Silicon Valley in recent years 
turns out to be 1.28%.° The single point expected value is $1 billion 
times 1.28% or $12.8 million. No wonder many 22 year olds enter 
risky entrepreneurial ventures against the odds when their next 
best alternative could be $50k a year working in a call center! But 
be careful: Single point expected value calculations are most use- 
ful when the underlying distribution is normal rather than skewed 
or long tailed. You check that by looking at the range, and whether 
the median and mean of the distribution are very different from 
each other. 


When the Australian government research organization CSIRO 
defended its WiFi intellectual property, it used a simple expected 
value calculation—but with a difference: It worked backward to the 
break-even probability of success, given its estimates of the costs 
of court action (§10m) and what they would receive if they prevailed 
($100m). The decision was made to pursue action in the courts 
because the board felt the chances of success were greater than 
an indifference probability of 10% ($100 million expected value of a 
successful court action divided by legal costs of $10 million). We will 
see this example more fully in the next chapter. 


Much has been made of Bayesian thinking in recent years with 
books like The Signal and the Noise.’ Bayesian thinking is really 
about conditional probability, which is the probability of an event 
given another event took place which also has a probability, called 
a prior probability. As a simple example, look at the probability 
of it raining given that it is cloudy (the prior probability), versus 
the probability of it raining if it is currently sunny. Rain can hap- 
pen in either case, but is more likely when the prior condition is 
cloudy. Bayesian analysis can be challenging to employ formally 
(as a calculation), because it is difficult to precisely estimate prior 
probabilities. But we often use Bayesian thinking when we think 


6CB Insights, May 25, 2015, www.cbinsights.com. 
Nate Silver, The Signal and the Noise (Penguin, 2012). 
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conditional probabilities are at work in the problem. In Chapter 6 
we look at the Challenger space shuttle disaster in the way a 
Bayesian statistician would approach the problem of a launch at a 
temperature not experienced before. 


Reasoning by analogy is an important heuristic for quick problem 
solving. An analogy is when you have seen a particular problem 
structure and solution before that you think may apply to your 
current problem. Analogies are powerful when you have the right 
reference class (that is, have correctly identified the structure 
type), but dangerous when you don’t. To check this, we typically 
line up all the assumptions that underpin a reference class and 
test the current case for fit with each. For example, our colleague 
Professor Dan Lovallo has looked at how you can use reference 
classes to predict movie revenues.® He takes a genre like drama, 
the presence of well known or little-known stars, and the budget, 
and then creates a model based on the reference class of mov- 
ies to compare with the one to be made. This can be surprisingly 
accurate and produce results better than a more complex 
regression model. 


There are limits to reasoning by analogy. For example: 


e When traditional business models in retailing, book publishing, 
and real estate were first challenged by disruptive Internet mod- 
els, there weren't useful prior analogies to draw on, to explain 
how the competitive dynamics would play out. 


e Inthe liquor market in Australia, a beer company acquired a wine 
company, reasoning that they were in the alcoholic beverages 
market and had common customers and distribution require- 
ments. The differences in product range, brand management, 
customer segments, and working capital requirements meant in 
practice they had few success factors in common. 


8Dan Lovallo, Carmina Clarke, and Colin Camerer, “Robust Analogizing and the 
Outside View: Two Empirical Tests of Case Based Decision Making,” Strategic 
Management Journal 33, no. 5 (2012): 496-512. 
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e Halo effects follow executives who have led turnarounds, 
assuming that their success will be replicated in new settings. 
“Chainsaw” Al Dunlap had such a reputation based on a turn- 
around at Scott Paper, but was fired in his new role at Sunbeam 
less than two years into the role.’ 


Break-even point. Every start-up company Charles and Rob see 
likes to talk about their cash runway—the months before cash runs 
out and they need a new equity infusion. Not enough of them really 
know their break-even point, the level of sales where revenue covers 
cash costs. It’s a simple bit of arithmetic to calculate, but requires 
knowledge of marginal and fixed costs, and particularly how these 
change with increased sales volume. The break-even point in sales 
dollars or units equals fixed costs/unit price less unit variable costs. 
Typically, the unit price is known. You can fairly quickly calculate the 
costs associated with each sale, the variable costs. The tricky part is 
how fixed costs will behave as you scale a business. You may face 
what are called step-fixed costs, where to double volume involves 
significant investment in machinery, IT infrastructure, or sales chan- 
nels. Knowing the break-even point and cost behavior brings insight 
about many aspects of a business, as we saw with the Truckgear 
pricing example in Chapter 1. 


As a real-world example of this, Charles used to ask McKinsey job 
applicants why car rental firms don’t charge your credit card if you 
don’t show up within the 24-hour window, but hotels do? This is a 
good example of rule of thumb thinking with high and low fixed 
cost businesses. It turns out that rental car firms have lower fixed 
costs and a much higher variable cost percentage than hotels, 
partly because they pay insurance and a number of other costs in 
a variable way (unlike regular consumers). Hotels, by contrast, have 
relatively low variable costs (room cleaning) per customer and large 
fixed costs. Rental car agencies also face many more substitutes 
(buses, trains, Uber, taxis) than hotels (Airbnb nowadays, but in 
the past really just a friend’s couch), and experience greater spa- 
tial competition in airports (they are much easier to price shop). 
By using some simple heuristics to understand these differences, 
you can relatively quickly understand why hotel companies both 


?"'Chainsaw Al’ Axed,” CNN Money, June 15, 1998. 
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need to charge for no-shows, and why they have the market power 
to do so relative to rental car firms. 


Marginal analysis is a related concept that is useful when you are 
thinking about the economics of producing more, consuming 
more, or investing more in an environment of scarce resources. 
Rather than just looking at the total costs and benefits, marginal 
analysis involves examining the cost or benefit of the next unit. 
In production problems with fixed costs of machinery and plant, 
marginal costs (again, the cost of producing one more unit) often 
fall very quickly—favoring more production—up to the point at 
which incremental machinery is needed. We add units until the 
marginal benefit of a unit sale is equal to the marginal cost. 


We can apply this idea to our hotel problem above. Imagine that 
the total fixed costs (mortgage, core staff, real estate taxes) of a 
200-room hotel are $10,000 per night, or $50 per room. If a potential 
guest shows up at 6 p.m. and there is a free room, should the hotel 
accept his offer of $30 for the night? We know the average cost is $50, 
but the marginal cost to take one more customer is tiny (probably 
close to zero, given even cleaning staff are already on duty), so they 
should take her offer. These economics are what has encouraged 
new internet businesses in last minute hotels and flights. 


Similarly, imagine a business that covers employee healthcare costs 
is considering subsidizing gym membership, which costs $400 per 
person. They believe that employees who go to the gym can save 
the company as much as $1000 per person in healthcare costs, and 
assume that if they pay a membership the team member will go. 
Should they do it? You are guessing the answer already: It depends 
on what percentage of staff members is already paying for their 
own gym memberships. In this case the marginal analysis tells us 
the company indifference point is 60% already paying their own 
way. If fewer make regular use of the gym without subsidy, it makes 
sense to offer the program (assuming the assumption of use and 
health benefits holds), but if more are already paying their own way 
(and getting the company a free benefit from lower health costs), 
they should spend their benefit budget on something else. 


This problem was suggested by Barry Nalebuff of Yale University. 
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As a final heuristic, consider the distribution of outcomes. Com- 
panies planning large projects often add contingencies of 10% 
or more for cost overruns, which can run into millions of dollars. 
Recently we saw data that for infrastructure megaprojects costing 
$1 billion or more, some 90% go over budget by 20% in the case 
of roads, and 45% in the case of rail projects."' In these cases the 
expected overspending is not in line with the contingency of 10%, 
but closer to 30%, almost a full standard deviation higher for cost 
overruns. These errors are large enough to wipe out the profitability 
of the projects. Take a look at the distributions shown in Exhibit 5.3. 


WHAT DISTRIBUTION OF OUTCOMES IS LIKELY? 
PROJECT COST DISTRIBUTION 


Tupical projects: costs + , 
Projects over ŠI billion 


10% contingency 


Road projects Mean 


10% at or below 


budget N 
T T T T T T T 
-30 -20 -10 0 10 20 30 -30 -20 -10 0 +10 +20 +30 
Standard Deviation Standard Deviation 
EXHIBIT 5.3 


"Nicklas Garemo, Stefan Matzinger, and Robert Palter, “Megaprojects: The Good, 
the Bad, and the Better,” McKinsey Quarterly, July 2015 (quoting Bent Flyvberg, 
Oxford Said Business School). 
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We see many common errors that relate to the distribution of out- 
comes. These include placing too much emphasis on the mean 
outcome, typically called the base case, and insufficient weight on 
outcomes that are one or even two standard deviations from the 
mean in a normal distribution. In the case of large projects, asking 
“Is the worst case bad enough?,” (as Kahneman and his co-authors 
put it) when it can turn out to be the true expected case!'* You 
have to bring history, analysis, your own experience, and some- 
times the experience of others through reference classes, together 
with judgment to estimate the shape of the project cost distribu- 
tion. Simulations become a valuable tool to highlight the range of 
outcomes possible. They are discussed in Chapter 6. 


Question-Based Problem Solving 


Once you have roughed out the scale and direction of your problem 
levers by using heuristics, it is time to dig deeper into the analysis. 
But not all the analysis you will do requires huge number crunching. 
Often you can get to good understanding and preliminary solutions 
with a straightforward Sherlock Holmes framework. 


We have found that the Sherlock Holmes approach of painting a 
picture of the problem by asking who, what, where, when, how, 
and why is a powerful root-cause tool to quickly focus problem solv- 
ing. We touched on the powerful role of questions in problem solv- 
ing in the previous chapter, not simply to clarify the problem under 
scrutiny but in defining choices and how they are to be evaluated. 


As we'll show below, we can often get to rough-cut solutions in as 
few as three questions. These examples illustrate the utility of pos- 
ing simple questions to advance good problem solving. For exam- 
ple, when we were discussing Charles's vote for the school levy in 
Chapter 1, he asked who were the best performing schools, what 
explained their better student outcomes, and why teaching quality 
was so important. Three question shortcuts can be represented in 


Daniel Kahneman, Dan Lovallo, and Olivier Sibony, “Before You Make that Big 
Decision,” Harvard Business Review, June 2011. 
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a decision-tree format; we show two examples in healthcare, one 
at the institution level and one at the individual level. The institu- 
tion example is how hospitals classify heart-attack patients. The 
individual example is whether or not to have a knee arthroscopy. 


Example: How to classify heart attack patients 


Hospitals have to make quick decisions to help suspected heart 
attack victims. What the hospital staff needs to decide is into what 
category to put incoming heart-attack patients, whether they are 
high risk or low risk. High risk requires much closer monitoring and 
allocation of staff. In a prior study, it was found that hospitals can 
segregate risk levels by asking three questions: whether the mini- 
mum systolic blood pressure over the initial 24 hours is greater than 
91, whether the age of the patient is greater than 62.5 years, and 
whether a condition known as sinus tachycardia is present.'? The 
data confirmed this categorization and can be represented in the 
decision tree structure of Exhibit 5.4 


INCOMING HEART ATTACK PATIENT CLASSIFICATION 
Risk 


Is the minimum High 
systolic blood No 


pressure over the 


High 
initial 24-hour period ie EE Yes "9 
greater than 91 ? Is the patient Yes tachycardia 
aged > 625° present? DS 
Low 
No 
Low 


EXHIBIT 5.4 


"Gerd Gigerenzer, Peter M. Todd, and the ABC Research Group, Simple Heuristics 
That Make Us Smart (Oxford University Press, 1999) (drawing on a medical study 
at the University of California, San Diego Medical Center by Breiman et al., 1993). 


Question-Based Problem Solving 


What we especially like about this example is how you start with 
hypotheses, test them with analysis, and then arrive at an induc- 
tive logic conclusion. It does a really nice job of helping decide 
how to quickly segregate high- and low-risk patients suspected 
of heart attacks. We note that this isn’t a new finding and may 
have been overtaken by newer approaches such as machine 
learning algorithms that are increasingly important in patient 
classification. 


Example: Rob’s bad knee 


SHOULD | HAVE A KNEE ARTHROSCOPY? 


Charles 
Yau £ fob F TA ` 7 
NÝ D ENZ =o $, 
(A R HA LF a% 
) 4 IAPR Pe P 
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Rob had a successful arthroscopy on his left knee 20 years ago and 
was able to compete in 10k and half marathon races each year. 
When he started to get inflammation in his right knee recently, 
which restricted his training schedule, he unsuccessfully tried 
icing and rest, and then had a meeting with a sports medicine 
professional. 


Every runner's knees are a focal point and the subject of important 
decisions following injury. The authoritative New England Journal 
of Medicine (NEJM) noted that over 700,000 APM (arthroscopic 
partial meniscectomies) knee procedures are carried out in the 
United States each year.'* When the diagnosis is meniscus tear, the 
runner is often presented with a recommendation to have an APM 
as the next step. 


“R. Sihvonen et al., “Arthroscopic Partial Meniscectomy versus Sham Surgery for 
a Degenerative Meniscal Tear,” New England Journal of Medicine 369, no. 26 
(December 26, 2013): 2515-2524. 
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What should Rob do? He had four options: 


1. Have an APM surgery performed by a sports medicine 
professional. 


2. Gather more information on probability of success for APM sur- 
gery in his particular case, planning a go/no-go decision after. 


3. Wait for new technologies in combination with physiotherapy 
and rehabilitation. 


4. Receive physiotherapy and rehabilitation. 


In Rob's case the sports-medicine professional assessed his condi- 
tion as being low discomfort, and degenerative with age and years 
of running. The sports medicine professional proposed a conserva- 
tive treatment approach, one that didn’t involve APM surgery. On the 
other hand, he learned from the sports medicine specialist of cases 
of football players with an average age of 22 with knee injuries who 
were going in for surgery immediately. He had also heard of research 
suggesting that APMs led to no better outcome than physiother- 
apy, and that he might be better waiting for stem cell treatments or 
printable hydrogel menisci, new technologies in development. 


How would you make this decision faced with so many disparate 
pieces of information? As in most problem solving you need to 
gather some facts. In Rob's case, he read a Finnish study in The 
New England Journal of Medicine that indicated that when there 
is low discomfort and age-degenerative meniscal tear, there is no 
significant improvement from APM compared to physical therapy. 
The research method involved a particular kind of randomized con- 
trol trial termed a sham control, where participants weren't aware 
of whether they had the procedure or not. The conclusions of the 
12-month follow-up were that the outcomes for the APM surgery 
were no better than for the sham APM surgery, notwithstanding that 
“both groups had significant improvement in primary outcomes.” 


'SR. Sihvonen et al., “Arthroscopic Partial Meniscectomy versus Sham Surgery for 
a Degenerative Meniscal Tear," New England Journal of Medicine 369, no. 26 
(December 26, 2013): 2515-2524. 


Question-Based Problem Solving 


Rob also wanted to assess whether he should wait for a new tech- 
nology solution to develop, such as stem-cell treatment or a print- 
able hydrogel meniscus. It turns out that clinical trials are getting 
underway for articular cartilage but not meniscal repair, so a reason- 
able conclusion can be drawn that any alternative to an arthroscopy 
is more than five years away.'® 3D printable hydrogel menisci are 
now getting a lot of interest but he couldn't estimate when they 
might be an option.” 


With the best facts and estimates in hand, things now started to 
fall into place. Rob decided the best option was to wait and see 
if a technology solution emerges while continuing to manage 
activity levels and the resulting inflammation in his right knee. 
The logic tree in Exhibit 5.5 shows both Rob's choices, and those 
faced by others with more discomfort. Of course they need indi- 
vidual opinions from specialists on the likelihood of APM surgery 
being successful given their circumstances. Finally, there are a lot 
of people who have a degenerative condition, who experience 
low discomfort, are not athletes, and are happy to adopt exercise 
regimes and physiotherapy as their path forward. 


The approach Rob developed is a three-question rule. In his case 
the answers led him to a course of action—to wait for new tech- 
nology while continuing with physiotherapy and rehabilitation 
(see Exhibit 5.5). 


In summary, we believe smart analysis starts with heuristics and 
summary statistics to assess the magnitude and direction of the 
key problem levers. When we have a good sense of these under- 
lying relationships, we often turn to questions like the examples 
above. This allows us to further refine our understanding and either 
make a decision based on available evidence, or to select more 
detailed answers. 


Cartilage repair via stem cells, Herbert Kim interview, UCSF, 2015. 
“David Nield, “New Hydrogel That Mimics Cartilage Could Make Knee Repairs 
Easier,” Science Alert, April 25, 2017. 
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EXHIBIT 5.5 


Root cause analysis is a problem solving tool that also uses ques- 
tions in a clever way. The technique of asking 5 Whys to get to 
the bottom of a problem was developed at the Toyota Motor 
Corporation.'® What is often termed a fishbone diagram is laid out 
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to visualize contributing factors to a problem (see Exhibit 5.6). 


Taichi Ohno, “Ask ‘Why’ Five Times about Every Matter,” Toyota Traditions, 


March 2006. 
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ROOT CAUSE ANALYSIS 


Root cause analysis (RCA) is a useful problem solving toolkit for some kinds of problems—and as 
a mindset or way of approaching problems. Most RCA tools were developed in complex industrial 
settings, and in the safety and accident investigation fields. The core idea is to dig deeper than the 
superficial or proximate causes of manufacturing faults or aviation accidents to uncover the source 
or root causes. You know you have a root couse when addressing that source removes the later 
problem. 


There are a number of specialist tools and processes employed in RCA, but three of them have good 
general problem solving utility: fishbone diagrams, 5-Whys, and Pareto Thinking (80-20 thinking, 
already covered above). When you have really complex processes, such as those we encounter in high 
tech manufacturing, it can be really difficult to isolate the sources of product defects and related 
problems. Fishbone diagrams, sometimes called by their Japanese name Ishikawa diagrams, can give 
you a comprehensive perspective on the potential sources of proximate and root causes. Good RCA 
processes use clear problem definition, careful disaggregation by time and stage of production, and 
team brainstorming on each potential source of error to tease out root causes 


ROOT CAUSE FISHBONE OR ISHIKAWA DIAGRAM 
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The 5 Whys are often used as above in production and operations 
problems where the technique was developed, and in developing 
safety cases. But the technique can be useful in other kinds of busi- 
ness problems too. The example in Exhibit 5.7 looks at a diagnosis 
of market-share loss, looking at superficial causes until the deeper 
source of the problem is found. 


Root cause analysis can also be applied to social problems such 
as homelessness. A recent Harvard Business Review article on how 
philanthropy is addressing deep-seated social problems made the 
remark that “donors don’t want to fund homeless shelters and food 
pantries. They want to end homelessness and hunger.”'? To do so 
means you first need to understand the homelessness problem and 
its various manifestations. Using the same technique as the exam- 
ple above we need to explore root causes by answering questions 
like these: 


e Is homelessness an issue mainly for men or women as well? 


e For women is it younger or older women who are disproportion- 
ately represented? 


e Are homelessness events for younger women episodic or long 
term in nature? If they are episodic, are they related to domestic 
violence or financial factors? 


e Are there adequate women’s shelters and support services in a 
particular community? 


e Are there programs in communities where domestic violence has 
been significantly reduced? 


The same train of questions applies to other parts of the homeless- 
ness logic tree, and most likely will produce different outcomes, 
including where the problems’ origins lie in mental health or drug 
or alcohol addiction, rather than financial distress or domestic vio- 
lence. We will revisit homelessness in Chapter 9, where we talk 
about how to tackle wicked problems. 


Susan Wolf Ditkoff and Abe Grindle, “Audacious Philanthropy,” Harvard Business 
Review, September—October 2017. 


Question-Based Problem Solving 


The 5 Whus approach, famous for use in Toyotas automotive production lines, is a powerful 
brainstorming tool for problem solving teams. It forces us to push beyond the local or contributory 
causes of problems toward root causes by asking why until there is no further possible why. Here 
is an example from a company that is losing market share but doesnt know the source of the 
problem. By forcing greater and greater specificity, the ultimate source of customer loss is found 
in inadequate training of phone customer service agents 


ROOT CAUSE OF MARKET SHARE LOSS 


Why? Why? Why? Why? Why? 
Is it because 
of price? 
Across all ak Product 
S products? feature issue? 
Is it existing Sense of oe 
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or 
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Not attracting 
new customers? 


This is the root cause of 
the share loss: Poor training 
for the phone teams and too 

much turnover 


While RCA tools were developed to find the sources of problems retrospectively, a root cause 
mentality can also be a forward-looking tool for your problem solving 


EXHIBIT 5.7 
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Chapter 5 Takeaways 


Start all analytic work with simple summary statistics and heuris- 
tics that help you see the size and shape of your problem levers. 
Don't gather huge data sets or build complicated models before 
you have done this scoping reconnaissance with rules of thumb. 
Be careful to know the limitations of heuristics, particularly the 
potential for reinforcing availability and confirmation biases. 
Question-based, rough-cut problem solving can help you 
uncover powerful algorithms for making good decisions and 
direct your empirical work (when required). 

Root cause and 5-Whys analytics can help you push through 
proximate drivers to fundamental causes in a variety of problems, 
and not just limited to production and operations environments. 


Problems to Try on Your Own 


1. 


How would you go about preparing estimates for whether Syd- 
ney Airport has a capacity shortage from the case in Chapter 1? 


. What is the break-even point for Truckgear from Chapter 1? 


Should they focus on margin or market share? 


. Try reasoning by analogy about the next new thing in mobile 


devices. What are the biases you might be bringing into this? 
How can you avoid the Walkman problem (not seeing some- 
thing that didn’t exist before)? 


. Make a forecast of something you're interested in. It could be 


the economy, fashion, sports, or entertainment. Set out what 
you have to believe for the forecast to be true. Then ask whether 
it’s likely to happen. Check out fivethirtyeight.com, where they 
do this for a living. 


. The blockchain is now 10 years old and a promising technol- 


ogy for smart contracts and governance. Draw out the diffu- 
sion curve for blockchain and explain how you validated your 
assumptions. 


Problems to Try on Your Own 


. Take a business plan in a tech venture or social enterprise you 
have seen and do a knock-out analysis, followed by a set of 
questions to the entrepreneur about what you have to believe 
for success. 


. What are the three questions that will help you decide whether 
to buy an electric car in the next two years? 


. Can you think of an example from your work where you have a 
problem you can interrogate with 5 Whys? Lay out the tree and 
sequence of questions. 


. How would you take the root cause case of homeless women to 
the next stage of inquiry? What are the second- and third-order 
questions to be asked? What analysis would you want to see 
undertaken before you felt comfortable with policies to address 
the issue? 
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Big Guns of Analysis 


x 


In Chapter 5 we introduced a set of first-cut heuristics and root 
cause thinking to make the initial analysis phase of problem solv- 
ing simpler and faster. We showed that you can often get good- 
enough analytic results quickly for many types of problems with 
little mathematics or model building. But what should you do when 


< 
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faced with a complex problem that really does require a robustly 
quantified solution? When is it time to call in the big guns—Bayes- 
ian statistics, regression analysis, Monte Carlo simulation, rand- 
omized controlled experiments, machine learning, game theory, 
or crowd-sourced solutions? This is certainly an arsenal of analytic 
weaponry that many of us would find daunting to consider employ- 
ing. Even though your team may not have the expertise to use 
these more complex problem solving tools, it is important for the 
workforce of today to have an understanding of how they can be 
applied to challenging problems. In some cases you may need to 
draw on outside experts, in other instances you can learn to master 
these techniques yourself. 


The critical first step is to ask this question: Have you adequately 
framed the problem you face, and the hypothesis you want to test, 
so that it’s clear you do need more firepower? If you decide your 
problem does fall into the genuinely complex range, you then need 
to consider other questions. Is there data available to support using 
an advanced analytic tool? Which tool is the right one for your prob- 
lem? Is there user-friendly software available to help your team use 
some of these tools? Can you outsource this aspect of the problem 
solving work? 


Firepower Availability 


It used to be that larger problems involved specialist-managed, 
time-intensive, and costly analytic processes. With the rapid rise in 
available computing power and the decline in data storage costs, 
more of these tools are now accessible. Software packages that 
allow you to run sophisticated analyses with only a few command 
lines are becoming more common, and have become substantially 
more straightforward for the everyday user. For example, regres- 
sion analysis is available in Excel by loading the Analysis ToolPak, 
and Monte Carlo simulation and other tools are available from the 
R-Project, a free analytic software resource. 


Given greater accessibility, we are seeing increased use of sophis- 
ticated modeling tools, such as machine learning, earlier in the 


Sequence Your Thinking 


problem solving process in businesses, consulting firms, and gov- 
ernment institutions. Sometimes this is warranted, but often the 
use of high-powered tools seems to be premature or misplaced. 
As we saw in Chapter 5, before embarking on complex analyses, we 
believe you are better off refining your thinking about the problem 
structure and the hypotheses you're testing by doing some initial 
order-of-magnitude analysis. Why? First-cut data analysis often 
points to direction of causality and size of impact, which are critical 
to evaluating the results of complex models later. Showing model 
correlations in the absence of clear explanation doesn’t prove any- 
thing. And knowing that one configuration or algorithm explains 
slightly more variance in a multivariable model often brings you 
no closer to understanding root causes. Data-fishing expeditions 
or unfocused analysis that “boil the ocean” are likely to result in 
inefficient problem solving. 


Professor Stephen Roberts, a machine-learning expert at Oxford, 
emphasizes the need to rigorously frame a question or hypothesis 
before embarking on complex analysis. He advises students not to 
commence machine-learning analysis until such time as they have 
developed a clear idea of the structure of the model and testable 
hypotheses, the essence of the scientific method.' 


In this chapter, we explain where and how to apply advanced ana- 
lytical approaches, as well as some of their limitations. Even if you 
don't personally employ the analytic big guns, you are likely to be 
presented with analysis that comes from these approaches. 


Sequence Your Thinking 


The process we recommend for deciding when to use the big 
guns is a sequential one. Our preferred sequence, not surpris- 
ingly, is to start with clearly defining the problem and forming initial 
hypotheses. Then get to know your data by looking at the mean, 
median, and mode, as well as other summary statistics. Plotting the 
distributions of key variables will point to the skew (difference from 


‘Personal communication with Professor Stephen Roberts, Oxford University. 
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a normal distribution), if any, in the data. Experiment with visu- 
alizing the data and correlations with scatter plots or hot-spot 
diagrams (Exhibit 6.2, which shows the air quality and asthma in 
London, is a good example of this). Of course this assumes you 
have data—we’ll show some approaches to dealing with limited 
data environments below. 


Which Big Gun to Choose? 


We worked with Oxford academics and our research team with 
experience in data analytics to design a decision tree that can 
help guide you to using the right analysis tool. The most important 
defining question at the outset is to understand the nature of your 
problem: Are you primarily trying to understand the drivers of cau- 
sation of your problem (how much each element contributes and 
in what direction), or are you primarily trying to predict a state of 
the world in order to make a decision? The first question leads you 
mostly down the left-hand branch into various statistical analyses, 
including creating or discovering experiments. The second ques- 
tion leads you mostly down the right-hand side of the tree into 
forecasting models, the family of machine or deep learning algo- 
rithms, and game theory. Some problems have elements of both 
sides, and require combining tools from both branches of the deci- 
sion tree. And simulations and forecasting models can be found on 
both sides of the tree (see Exhibit 6.1). 


When you are focused on understanding the complex causes of 
your problem, so that you can develop strategies for intervention, 
you are uSually in the world of statistics. Here the quality and extent 
of your data is critical to knowing which tool you can use. If you have 
good data, you can use correlation and regression tools, many of 
which are available in quite easy-to-use software packages. There 
are more and more externally available data sets, many of which 
are low cost or free. Sometimes external data doesn’t exist for your 
problem, but you can design an experiment to develop your own 
data. Experiments have long been so important in safety-critical 
fields like medicine, for instance, where robustly inferring whether 
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drugs cause intended effects and side effects is essential. Experi- 
ments have become ubiquitous in business and economics as meth- 
odologies improve and costs to run experiments have declined, 
particularly in online settings. In lots of cases, however, there are 
costs, complexity, and ethical hurdles to experimentation (which 
involve treating two similar groups differently). If you are clever (and 
lucky) you may be able to find a natural experiment where the struc- 
ture of some real-world situation provides its own control group. 
You may also find yourself in a situation where you have only partial 
data and no potential for experiment. Here Bayesian (conditional) 
statistics can be helpful. 


When you are mainly focused on predicting outcomes or poten- 
tial states of the world to plan decisions, you first have to deter- 
mine if it is important to anticipate and model the reaction of other 
players or competitors. If you do, you will probably be employing 
game theory, perhaps enhanced with statistical knowledge. If you 
are not focused on competitor strategies, you are probably opti- 
mizing some kind of system or forecasting uncertain outcomes. If 
the first of these, you may be learning how to employ tools from 
machine learning (where understanding is less important that accu- 
rate prediction). Particular problems around image recognition and 
language processing may be best suited to the sub-field of neural 
networks. If the second, you will likely be employing forecasting 
models or simulations. 


Case Studies for Employing the Big Guns 


We know this is a complicated decision tree. To illustrate each of 
these analytic tools in action, we provide case examples of how 
they are used in problem solving. We start with simple data analysis 
and then move on to multiple regression, Bayesian statistics, simu- 
lations, constructed experiments, natural experiments, machine 
learning, crowd-sourced problem solving, and finish up with 
another big gun for competitive settings, game theory. Of course 
each of these tools could warrant a textbook on their own, so this 
is necessarily only an introduction to the power and applications of 
each technique. 
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Summary of Case Studies 

1. Data visualization: London air quality 

Multivariate regression: Understanding obesity 

Bayesian statistics: Space Shuttle Challenger disaster 
Constructed experiments: RCTs and AJB testing 

Natural experiments: Voter prejudice 

Simulations: Climate change example 

Machine learning: Sleep apnea, bus routing, and shark spotting 


Crowd-sourcing algorithms 


so oN DO 0 e Ww DN 


Game theory: Intellectual property and serving in tennis 


It is a reasonable amount of effort to work through these, but bear 
with us—these case studies will give you a solid sense of which 
advanced tool to use in a variety of problem settings. 


Data Visualization: Clusters and Hotspots—London 
Air Quality Example 


Publicly available data sets are becoming more widely accessible. 
To illustrate the power of relatively simple analysis on public data 
sets, we take the case of air quality in London (Exhibit 6.2). We know 
that one of the major factors affecting air quality is small particu- 
late matter, PM 2.5 and PM 10. You don’t want to live in places 
where PM 2.5 levels are frequently high because of respiratory 
and cardiovascular impact. We looked at London using air qual- 
ity data and asthma hospital admissions by postcode for the year 
2015. The heat map that results shows the neighborhoods with the 
highest level of risk. As a first cut, it suggests exploring the issue 
further is warranted, even though the correlations aren't especially 
high between particulate matter and hospital admissions for year- 
long data. And as we know, correlations do not prove causation; 
there could be an underlying factor causing both PM 2.5 hotspots 
and asthma hospital admissions. Experiments, more granular data 
analysis, and large-scale models are the next step for this work. 
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The air that we breathe is key to our health. It was a factor in Charles's analysis of 
where to live. In this case we zero in on air quality and what it implies for where to 
live in London. 


Rob had noticed a Public Health England report a few years ago concluding that 3000 
people in London died in 2010 as a result of PM... particles in the air that affect 
respiratory health including asthma and is correlated to heart and lung disease. A 
number of studies have concluded that there is really no safe level of exposure to 


PM... 


We asked William Rathje, a Rhodes scholar on our team with data analysis and machine 
learning skills. to spend an hour on the question of whether we could identify 
hotspots for asthma and PM... levels. William produced these remarkable pictures 
below. based on 2015 data, that show where the asthma hotspots are. The left image 
is measured by accident and emergency admissions, and the right is measured by PM. 
hotspots. The visual representation shows a significant overlap 
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The analysis shows what you can learn about things like air quality from modest 
commitments of time and data analysis skills. Wed argue that this is a non-trivial 
consideration when making a decision on where to live. 


EXHIBIT 6.2 


Source: Q. Di et al., “Air Pollution and Mortality in the Medicare Population,” New England 
Journal of Medicine 376 (June 29, 2017), 2513-2522. 
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Regression Models to Understand Obesity 


Obesity is a genuinely wicked problem to which we will return in 
Chapter 9. There are multiple explanations for its prevalence and 
increase, wide differences between communities, and complex 
behavioral and policy factors at work. Moreover, there are no suc- 
cess stories that we are aware of for communities of any scale 
reversing the trend. In this example we want to highlight regression 
analysis as a powerful analytic tool to understand underlying drivers 
of the problem of obesity. It doesn't solve the obesity conundrum, 
but it shows us where to look for solutions. 


We asked one of our research assistants, Bogdan Knezevic, 
a Rhodes scholar doing a PhD in genomics and big data analyt- 
ics, to test several hypotheses related to obesity at the city level 
using regression analysis. The hypotheses were developed from 
the comprehensive McKinsey Global Institute (MGI) study on obe- 
sity.2 Bogdan gathered data on 68 US cities, including obesity 
prevalence,? educational attainment, median household income, 
city walkability,* and climate comfort score.” The climate comfort 
score is the suitability of weather to physical activity. It is calculated 
as the sum of temperature and relative humidity divided by four. 
Comfort scores are like Goldilocks’s porridge: You don’t want it 
too hot or too cold but just right with temperature and humidity. 
The data showed that education, income, walkability, and comfort 
score are all negatively correlated with obesity. Bogdan found that 
all the variables except walkability were individually correlated with 
obesity. Perhaps, surprisingly, the comfort index and walkability 
had almost no correlation with each other. Other variables, espe- 
cially income and education, are highly correlated with each other 


?MGI Obesity Study, 2013. 

3American Fact Finder, https://factfinder.census.gov/faces/nav/jst/pages/index 
xhtml. 

“Tim Althoff, Rok Sosic, Jennifer L. Hicks, Abby C. King, Scott L. Delp, and Jure 
Leskovec, “Large-Scale Physical Activity Data Reveal Worldwide Activity Inequal- 
ity,” Nature 547 (July 20, 2017): 336-339. 

"Sperling's Best Places, www.bestplaces.net. 
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(68%), which can make sorting out relative influence in causation 
more difficult.® 


In Exhibit 6.3, individual cities are compared for obesity prevalence 
measured by BMI (body mass index) and household income, with 
comfort scores shown as circles. The correlation between obesity 
and income is clear and large: Income accounts for 71% of the vari- 
ance in obesity between cities. The line of best-fit shows that hav- 
ing household income at $80,000 versus $60,000 is associated with 
a seven-percentage-point reduction in obesity, over 30% percent. 
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EXHIBIT 6.3 


‘The test for multicollinearity, the extent to which one independent variable could 
be linearly predicted from another independent variable, were positive for income 
and education, while others returned negative correlations between the two; both 
variables were retained. 
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The best model for capturing obesity differences between cities 
depended ona combination of income, education, comfort, walk- 
ability, and an income/education interaction term that accounts 
for the relationship between those two variables. All the variables 
were statistically significant and negatively correlated with obe- 
sity. Some 82% of the variance in obesity can be explained when 
all four variables are included, a level of explanatory power that is 
relatively high in our experience. Nonprofits, such as the Robert 
Wood Johnson Foundation, have noted how significant income 
and education factors are in explaining obesity differences in the 
United States. 


Multiple regression analysis allows us to combine and control for 
variables as we explore our understanding of the underlying rela- 
tionships. For instance, running a linear regression on our data using 
only walkability leads us to conclude that there is no significant cor- 
relation between city walkability and obesity rates. However, when 
we put both walkability and a comfort score together using multi- 
variable regression, we see that walkability is significantly correlated 
with obesity after controlling for weather. 


This example is just a simple one to show how regression analysis 
can help you begin to understand the drivers of your problem, and 
perhaps to craft strategies for positive intervention at the city level. 


As useful as regression is in exploring our understanding, there are 
some pitfalls to consider: 


e Be careful with correlation and causation. Walkable cities seem 
to almost always have far lower obesity rates than less walkable 
cities. However, we have no way of knowing from statistics alone 
whether city walkability is the true cause of lower obesity. Per- 
haps walkable cities are more expensive to live in and the real 
driver is higher socioeconomic status. Or perhaps healthier peo- 
ple move to more walkable communities. 


e Regression models can be misleading if there are variables that 
we may not have accounted for in our model but that may be 
very important. This model is configured at the city level, and so 
is blind to individual level behavioral or cultural factors. 
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e Adding more variables may improve the performance of the 
regression analysis—but adding more variables may then be 
overfitting the data. This problem is a consequence of the under- 
lying mathematics—and a reminder to always use the simplest 
model that sufficiently explains your phenomenon. 


Bayesian Statistics and the Space Shuttle 
Challenger Disaster 


For those who lived through the Space Shuttle Challenger dis- 
aster, it is remembered as an engineering failure. It was that of 
course, but more importantly it was a problem solving failure. 
It involved risk assessment relating to O-ring damage that we now 
know is best assessed with Bayesian statistics. Bayesian statistics 
are useful in incomplete data environments, and especially as a 
way of assessing conditional probability in complex situations. 
Conditional probability occurs in situations where a set of prob- 
able outcomes depends in turn on another set of conditions that 
are also probabilistic. For example the probability of it raining will 
vary depending on whether there are clouds in the sky, which itself 
is probabilistic. 


Lets look at our Challenger space shuttle example from the Intro- 
duction again to see how this analytic approach can help you 
assess risk. O-ring failure under low temperatures was the prob- 
able cause of the Challenger space shuttle erupting shortly after 
lift-off.’ Rubber O-rings were used on the Challenger to prevent hot 
gases from leaking and were designed to compress and expand 
with temperature. However questions were raised about the resil- 
iency of cold O-rings given the unusually low temperature at launch 
of 31 degrees Fahrenheit, some 22 degrees below the minimum 
on previous launches, coupled with the finding by incident inves- 
tigators that a compressed O-ring is five times more responsive at 
75 degrees than at 30 degrees Fahrenheit. In this case the prob- 
ability of O-ring failure is (partially) conditional on the probability of 
experiencing a particular temperature range. 


7Presidential Commission on the Space Shuttle Challenger Accident, 1986. 
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Researchers have relooked at the circumstances of the O-ring 
failure and reached conclusions that substantiate the focus on 
O-rings but relate specifically to the data analysis and problem 
solving undertaken prior to launch on January 28, 1986.8 They argue 
that if Bayesian statistics had been employed, and correct sampling 
done, this near inescapable conclusion would have been reached: 
that the probability of failure, given a predicted launch temperature 
of 31 degrees Fahrenheit, was a near certainty. 


What made the analysis difficult for engineers at the time was 
that the data they had was for flights where the temperature at 
launch was in the range of 53 to 81 degrees Fahrenheit, when in 
this instance the launch of Challenger was to occur at an unusually 
cold 31 degrees Fahrenheit. They looked at the temperature when 
O-ring damage occurred and noted the level of O-ring thermal dis- 
tress that occurred between 53 degrees and 75 degrees. They did 
look at limited data below this temperature range, but couldn't see 
a clear pattern (Exhibit 6.4). The engineering team reported this 
conclusion to NASA prior to the launch to address concerns that a 
low launch temperature could affect O-rings. 


FLIGHTS WITH DAMAGED O-RINGS & TEMPERATURE 
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8C. J. Maranzano and R. Krzystztofowicz, “Bayesian Re-Analysis of the Challenger 
O-ring Data,” Risk Analysis 28, no. 4 (2008): 1053-1067. 
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The data they should have been looking at, however, was the data 
on all flights in relation to temperature and O-ring damage. What 
becomes clear from the data below, when you include all flights 
with no O-ring damage incidents, is a different picture: For tem- 
peratures below 65 degrees, all 4 flights had incidents, or 100%! 
Above 65 degrees only 3 of 20 flights, or 15%, had damage inci- 
dents. With all the data, the relationship between temperature and 
O-ring performance becomes a lot clearer to see (Exhibit 6.5). 


FLIGHTS WITH DAMAGED O-RINGS & TEMPERATURE 
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Bayesian statistician Professor Sally Cripps of the University of 
Sydney, a former colleague of Rob’s at the Australian Graduate 
School of Management, has taken this data and combined it with 
a prior probability of failure of 30% (the failure of seven O-rings in 
23 flights). The resulting posterior probability of failure given launch 
at 31F is a staggering 99.8%, almost identical to the estimate of 
another research team who also used Bayesian analysis. 


Several lessons emerge for the use of big guns in data analysis 
from the Challenger disaster. First is that the choice of model, in 
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this case Bayesian statistics, can have an impact on conclusions 
about risks, in this case catastrophic risks. Second is that it takes 
careful thinking to arrive at the correct conditional probabil- 
ity. Finally, how you handle extreme values like launch tempera- 
ture at 31F, when the data is incomplete, requires a probabilistic 
approach where a distribution is fitted to available data. Bayesian 
statistics may be the right tool to test your hypothesis when the 
opportunity exists to do updating of a prior probability with new 
evidence, in this case exploring the full experience of success and 
failure at a temperature not previously experienced. Even where 
actual Bayesian calculations aren't employed, a Bayesian think- 
ing approach (taking account of conditional probabilities) can be 
very useful. 


Constructed Experiments: RCTs and A|B Testing 


Often data we would love to have to shed light on our problem 
just doesn't exist. Experiments give us a way to make our own data. 
There are lots of advantages to this—in particular, your competi- 
tors are sure not to have your data. Let's take a look at two types 
of experiments that have become popular in the corporate world. 


Randomized controlled experiments allow us to test a change in 
one variable while controlling for all other variables. As we saw in 
the obesity example above, maybe it’s not a feature of walkable cit- 
ies themselves that makes people less obese. Maybe people who 
are already healthy tend to move to cities that are walkable because 
they like to walk. Experiments avoid this kind of potential mistake in 
understanding causality. But they’re often hard to execute in prac- 
tice. We can't easily make a city more pedestrian friendly and com- 
pare it to another city. But when we can set up a good experiment, 
they can have great explanatory power. 


Experiments can be useful for evaluating different types of inter- 
ventions. For example, a randomized-controlled experiment can 
help you test whether a specific new program offering individuals 
incentives to walk five miles per day reduces obesity better than 
an existing program, controlling for all other factors. Randomized 
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controlled trials (RCTs) require you to recruit a pool of participants, 
and randomly allocate participants into two or more treatment 
groups and a control group. In this case, the treatment is the new 
exercise incentive program. They work particularly well when you 
want evidence that a specific intervention is the likely cause of a 
change in outcome. By randomly allocating the treatment and 
control groups, you ensure that all other characteristics of this 
group—like demographics, general health status, baseline exer- 
cise rates, and diet—vary only at random. When your sample size 
is large enough, no one group should be unbalanced compared 
to the general population. For example, when assigned randomly, 
your exercise-treatment group won't have more women than men 
compared to your control group. If your treatment group improved 
more than the control, by a statistically significant amount, you 
know that your exercise program, not gender or any other vari- 
able, is a strong candidate for explaining the improvement. It 
doesn't entirely remove so-called confounding factors, but it’s a 
good start.’ 


Special types of experiments are variations on randomized con- 
trolled trials. Market trials are one such special case of experimen- 
tal methods, where a new product is tried on a small test market 
before a full roll out. One of the most frequently used experimen- 
tal techniques in business today is A/B testing. A/B testing is used 
to make adjustments to product offers in real time. For instance, 
you may want to test the effect of changing the background color 
of your website homepage on the length of time visitors stay on 
your website. An experiment is quick and easy to perform and 
provides evidence for one design over another. An A/B test would 
involve randomly selecting two groups of visitors to your website 
and showing one group (group A) your webpage with the new 
design while the other group (group B) of users, the control group, 


°The Book of Why by Judea Pearl and Dana Mackenzie (Allen Lane, 2018) has a 
good example of confounding factors with the example of the correlation between 
Nobel Prize winners and per capita chocolate consumption of the country they 
come from. Wealthy countries invest more in education and eat more chocolate, 
so the wealth and location of prizewinners are confounders. 
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sees your old design. You track each group’s usage pattern, and in 
some cases follow up with a survey after they visit the site. You can 
then choose your website homepage color using real data about 
how the design, color, or other variations affect user retention 
times or e-commerce uptake. And because you ran a randomized 
experiment, you know that it’s more likely the design—not other 
variables—influencing user experience. 


A/B Testing 


Electronic Arts (EA) recently used an A/B test to explore the impact 
of sales and promotions on the sale of new game releases.” Driven 
by revenue targets coincident with the release of the next version 
of SimCity, SimCity 5, EA was surprised when the initial advertis- 
ing layout of the website with a promotional offer in a headline 
banner were not generating rising pre-orders. Could it be that the 
pre-order call to action was incorrectly placed? Their team con- 
structed several additional variations on the pre-order page and 
randomly exposed the site visitors to the different options, say A/B/ 
C/D. Option A, without any promotion at all, and Option B with a 
promotional banner, are shown in Exhibit 6.6. 


The team concluded that the site with no promotional offer led 
to 43.4% more purchases of the new game edition, a significant 
increase. Here A/B testing allowed them to experimentally explore 
their customers’ responses to sales and advertising. In the pro- 
cess they learned that SimCity customers’ decision to purchase 
was not contingent on a promotion. This kind of testing is particu- 
larly easy to do in online environments, but can also be employed 
in different cities or at the retail store level to generate similar 
results. Real-world experiments are harder when large numbers are 
required to create controls for confounding factors, as is often true 
in healthcare examples, or where it is unethical to expose people 
to experimental choices without their consent. This might lead us 
to look at so-called natural experiments. 


https://blog.optimizely.com/2013/06/14/ea_simcity_optimizely_casestudy/. 
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EXHIBIT 6.6 


Natural Experiments: Voter Prejudice 


Even if a question could be answered by a constructed experiment, 
sometimes you can’t run one. Your turnaround time or budget might 
be too limited, or ethical considerations might rule out experimen- 
tation. Governments, for example, often want to understand the 
effects of changing a tax or benefit program. But it is difficult (and 
often illegal) to adjust taxes or social benefits for some groups but 
not for others (the control groups) in a way that would shed light on 
the best course for public policy. How can you collect convincing 
evidence under such constraints? 


One answer is the natural experiment, also called a quasi- 
experiment: If you can’t run an experiment yourself, look to see if 
the world has already run it—or something like it—for you. Institu- 
tions often make choices that, by accident, create environments that 
can be much like an experiment. Many government programs rely 
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on essentially arbitrary cutoffs or, by some combination of history 
and happenstance, treat some geographic areas differently from 
other similar areas. This creates natural affected groups and control 
groups. In the United States, for instance, the national minimum 
drinking age of 21 creates a sharp change in the legal status of 
drinking between people aged 20 years and 364 days and 21 years 
and 0 days. When we can safely compare people just on one side of 
a policy's cutoff to those on the other side, or areas that were heav- 
ily affected by a given intervention to areas that were less affected, 
then we have a natural experiment that can tell us many of the same 
things an actual experiment could. 


Natural experiments can be powerful tools, giving us high-quality 
evidence in environments unsuitable for experimentation. But 
because they are, by definition, not actual experiments, natural 
experiments require more attention to context and potential con- 
founding factors that might damage the experimental character of 
the environment. To see how you might use a natural experiment 
to answer a question where actual experiments can’t go and that 
would stump our other problem solving tools, let's consider one 
example from Evan Soltas, a Rhodes Scholar, and David Broock- 
man, a political scientist at Stanford University (now published as a 
Stanford GSB Working Paper). Evan and David's question was: Do 
voters discriminate against minority candidates in elections? 


It's important, first, to see what methods won't work to answer this 
question. We can't simply gather data on white and nonwhite can- 
didates for office, calculate the average difference in the number 
of votes they receive, and declare whatever difference exists to be 
“discrimination.” There are potentially many differences between 
white and nonwhite candidates other than race; for instance, whites 
and nonwhites may run for different offices or campaign on differ- 
ent platforms. Without an experiment—or something very close to 
one—untying this knot, and distinguishing the independent effect 
of discrimination, is hopeless. 


Evan and David found their natural experiment in an obscure, 
unusual procedure in the Illinois Republican presidential primary. 
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In it, voters cast multiple ballots for delegates selected to represent 
presidential candidates at the Republican convention, where a 
candidate is formally nominated. An Illinois voter who supported 
Donald Trump, for instance, couldn't vote for him in the primary, 
but rather would vote for three delegates who are legally bound to 
attend the convention and support this candidate. The twists that 
create the natural experiment in this case are: first, that these dele- 
gates’ names appear on the ballot, even though their identities are 
nearly entirely irrelevant. Most delegates are political unknowns— 
so unnoteworthy that many aren't discoverable on Google, and 
certainly are not known to most voters. Second, voters don’t have 
to vote for all of their favored presidential candidate's delegates, 
and so delegates for the same candidate need not receive the 
same number of votes to effectively represent the candidate at the 
convention. 


Imagine a candidate nominated two presumably-white delegates, 
Tom and Dick, and a presumably-nonwhite delegate, Jose. To sup- 
port their candidate fully, his voters should cast ballots for Tom, 
Dick, and Jose. Some voters might subtly discriminate, however, 
by voting for Tom and Dick but not for Jose. If enough voters dis- 
criminate, then minority delegates like Jose will receive significantly 
fewer votes than non-minority delegates like Tom and Dick; that is, 
relative to delegates who are perfectly comparable in the sense 
that they face exactly the same pool of voters at the same time, and 
are bound to support the same presidential candidate, and differ 
only in their identity as indicated by their name on the ballot. This is 
a great natural experiment! 


In the terminology of natural experiments, one delegate is “treated” 
with a likely minority name and is compared against the control of 
two delegates without likely minority names. Evan and David's Illi- 
nois election data, in fact, contained over 800 such experiments. 
What they found was revealing about the extent of prejudice 
among American voters: The delegates with names that suggested 
they were nonwhite received about 10% fewer votes as compared 
to their respective white control delegates. 


NATURAL EXPERIMENT: 
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TESTING FOR PREJUDICE IN VOTING PATTERNS 
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Exhibit 6.7 shows how Evan and David went from research design— 
that is, the idea of their natural experiment—to their statistical 
analysis. While we could, by simple arithmetic, calculate the per- 
centage gap between the nonwhite and white delegates in each 
of Evan and David's experiments, a properly specified independ- 
ent variable regression analysis will do such a calculation for us. 
Standard statistical software, such as R, Stata, or various Python 
packages, can easily support regression analyses similar to Evan 
and David's. 


In this analysis, the crucial independent variable was the perceived 
race of delegates. To get the data, Evan and David were entrepre- 
neurial: They matched delegates’ last names to a dataset from the 
US Census that provides, for each last name, a share of Americans 
with a last name who are nonwhite. They also paid online workers on 
Amazon's Mechanical Turk (mTurk) marketplace to guess, without 
further information, the racial background of delegates from their 
names alone, replicating their view of what voters might be doing 
in the privacy of the voting booth. This is a form of crowdsourc- 
ing, which we'll discuss more below. The actual vote counts, their 
dependent variable, came from the Illinois State Board of Elections. 
Knowing where and how to find data, whether from administrative 
releases from the Census or from mTurk surveys, is itself a vital tool 
in your analytical arsenal. 


What can go wrong in natural experiments like Evan and David's? 
Natural experiments rely upon the assumption that treatment and 
control groups are comparable—whereas, in an actual experiment, 
this would be guaranteed by randomizing treatment. Evan and 
David had to rule out differences within groups of delegates like 
Tom, Dick, and Jose other than Jose's minority status. One poten- 
tial confound of this kind would be if minority delegates were less 
likely to hold other public offices than nonminority delegates. Then 
some voters might support Tom or Dick, but not Jose, because 
Tom is their mayor or Dick is their councilman, not because Jose's 
minority background makes them uncomfortable. A large part of 
the work Evan and David put into their project, Evan said, was in 
addressing such comparability concerns. 


Case Studies for Employing the Big Guns 


It doesn't mean that you can extrapolate from this analysis to other 
regions or voters; you only know that it’s true and present for the 
precise setting where the data was collected. 


Simulations: Climate Change Example 


Simulations are computer imitations of what is expected to hap- 
pen in the real world. They are powerful because they allow you 
to see many potential states of the world with different input 
assumptions. All you need is a model, some initial conditions, 
and a software program, such as Microsoft Excel, to implement 
a simulation. Simulations are often fast and low cost; if you have 
a well-configured model—and that is the hard part—they can allow 
you to test parameter changes without expensive real-world data 
collection. They can be useful in both branches of our analytical 
tools selection tree. 


Simulations are particularly helpful for forecasting the impact of 
policy or parameter change on an outcome of interest, for example 
of climate change or economic outcomes. Given your known model 
(and the source of uncertainty if you have it), changing the value of 
different independent inputs in the model can change the distri- 
bution of your results. This is called sensitivity analysis and could 
allow you to use simulations to test the effects of climate change 
on global mean temperatures, and then on economic outcomes at 
the county level, as recently highlighted in The Economist.'' The 
authors of the underlying paper, Hsiang et al, combined a range 
of temperature and other weather outcomes from climate models, 


"The Economist article referenced the work of Hsiang, Kopp, et al. (“Estimat- 
ing Economic Damage from Climate Change in the United States,” Science 
2017) that with each 1 degree Fahrenheit increase the cost of climate change for 
the United States would be an aggregate reduction of .7% of GDP. http://science 
.sciencemag.org/content/356/6345/1362.full?ref=finzine.com%20; “Climate Change 
and Inequality,” The Economist, July 13, 2017, https://www.economist.com/ 
news/finance-and-economics/21725009-rich-pollute-poor-suffer-climate-change- 
and-inequality; http://news.berkeley.edu/2017/06/29/new-study-maps-out-dramatic- 
costs-of-unmitigated-climate-change-in-u-s/. 
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with a set of economic models that predict outcomes at the county 
level under different climate scenarios, in a large-scale simulation of 
future impacts of climate change. 


The simulation model used by the authors incorporated 44 sep- 
arate climate change models, rainfall and temperature weather 
predictions, and uncertainty. They mapped their projections onto 
joint distributions of possible economic outcomes, exploring the 
impacts on labor, violent crime, mortality, rainfall, electricity con- 
sumption, temperature, and CO, at the county level in the United 
States (Exhibit 6.8). To calculate economic costs in terms of GDP, 
they monetized non-market impacts and then aggregated each 
county prediction to the country level. This analysis showed that 
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the negative impacts of climate change are likely to be dispropor- 
tionately felt by counties that are already poor. Climate-change 
analysis, like this one, exhibits one of the main strengths of simula- 
tions rather than actual experiments. In the case of climate, we can- 
not wait for global temperatures to increase to explore the effects, 
rather we want to simulate possible future effects reliably now in 
an effort to mitigate and prevent irreversible damage. Simulations 
work best when you have well configured and tested underlying 
models, and where your simulations are testing projected ranges in 
line with the underlying ranges of the original data. 


Machine Learning Solutions 


Machine learning techniques are at heart very similar in concept to 
traditional statistical tools like multiple regression. They often per- 
form much better than these traditional techniques when there is a 
large amount of data in the right format, and when there are more 
complex patterns and interactions in the data. These tools have 
caught on, particularly in the past several years, because comput- 
ing power has grown and is available at reasonable cost to manage 
large datasets and sophisticated learning algorithms. Learning a 
simple scripting language such as Python or R allows you to begin 
using state-of-the-art machine learning tools, and it is within the 
capabilities of many teams. 


Let's look at the problem of classifying whether an individual is at high 
or low risk of becoming obese. If you have large amounts of data— 
perhaps walking records from individuals’ mobile phones, demo- 
graphics, and health information—and information about what 
profiles in this population went on to develop obesity, you allow the 
machine learning algorithm to learn how to identify hallmark signs 
of obesity risk from the data. Unlike with linear regression, you're 
less concerned here about identifying and measuring the specific 
variables that indicate disease risk (although you can often infer 
these using machine learning) as you are about building a learning 
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model that can make accurate predictions. With these techniques 
you're often less involved in the learning process: Instead of select- 
ing which variables to use, you often allow the computer algorithm 
to select the correct combination of features. With the right amount 
of data, machine learning can often outperform traditional statisti- 
cal techniques for prediction. 


We now turn to examples of machine learning that are producing 
outcomes that save lives and resources. We have selected one in the 
field of medicine, one that relates to education, and one in the field 
of drone technology to illustrate the diversity of machine-learning 
applications in problem solving. 


Can a Computer Predict the Onset of Sleep Apnea Better 
Than Doctors Can? 


One of the reasons machine learning has gained so much traction 
is because of its demonstrated ability to make predictions that 
are better than human judgment in particular domains. Oxford 
doctoral students Logan Graham and Brody Foy, cofounders of 
the Rhodes Artificial Intelligence Lab, a student-led organization 
dedicated to solving social problems with machine learning, spear- 
headed the development of a machine-learning model that out- 
performed expert doctors at predicting whether children would 
go on to develop sleep apnea within 6 to 12 months from clinical 
examination. The tool used data from a simple questionnaire con- 
taining information on demographics, patient BMI, and co-morbid 
diseases like asthma and acid reflux to make predictions that con- 
sistently outperformed clinicians by upwards of 20% in accuracy. 
Logan speculates that the model might succeed by shortcutting 
biases in doctors’ decision-making processes or by considering rel- 
evant combinations of predictive features that medical practition- 
ers might overlook. The model they developed takes into account 
the general population while practitioners work patient by patient. 
The algorithm showed marked success even on a relatively small 
sample of just over 400 patient records. Conventional diagnosis of 
sleep apnea requires patients to undergo costly and invasive sleep 
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studies. A predictive model that reduces false positive screens 
could save patients and doctors considerable time and money. 


Logan notes that machine learning works very well for problems 
that involve predicting future outcomes. However, he also cau- 
tions that prediction problems require a dataset that matches 
the general population distribution. The sleep apnea data was 
drawn from patients who had already been flagged by their 
clinicians as being at risk of a sleep disorder. Because of this, a 
different and larger dataset would be required to make robust 
predictions if the tool were to be used on the general public. 
The important point here is that machine learning is predicting 
an outcome, not understanding the solution or how a decision 
is being made. 


Routing School Buses with Computers 


A group of MIT researchers developed an algorithm that saved 
the Boston Public School System $5 million by optimizing school 
bus routes for 30,000 students. Previously, school administrators 
designed bus routes by hand, a laborious process that took weeks 
every year. The MIT algorithm runs in minutes and eliminated a num- 
ber of routes to 230 different schools, which used to be serviced by 
650 school buses. 


Bus routing is an example of a problem where investing in the crea- 
tion of a computational tool upfront can save time and money in 
the long run. The algorithm automates the process of bus routing, 
a process that humans can do with their own intuition, but is so 
computationally intensive that it can be better served by relying 
on a mathematical process. It’s a problem well suited to automa- 
tion. The routing system will need to be run year after year, but the 
underlying decision-making engine need not change each time it 
is used. There's no real need for human decision making here, and, 
if anything, human oversight would hamper the decision process. 
We have objective external criteria that we can use to assess the 
effectiveness of the algorithm. 
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Shark Spotting from Drones 


Shark attacks are a real and perceived danger on many beaches 
around the world. Netting is often put in place to reduce the risk of 
shark attacks but at considerable financial cost and species loss due 
to entanglement. A solution is emerging on Australian beaches by 
a company called the Ripper Company that Rob is involved with, 
a company we shall discuss in more detail in Chapter 8. The solution 
involves the combination of camera-carrying drones and machine 
learning. The inspiration for shark spotting via machine learning 
came from Australian publisher Kevin Weldon, who was the first 
president of International Surf Life Saving, and Paul Scully Power, 
Australia’s first astronaut. 


Here's how it works. Drones patrol the beach and transmit video to 
the operations center. Together with machine learning experts at 
the University of Technology Sydney (UTS), the Ripper Company 
developed a shape-interpreting algorithm, the first of its kind, 
which predicts the presence of a shark, as distinct from porpoises or 
humans. When a shark is detected by the algorithm and confirmed 
by an operator, the beach patrols alert swimmers and clear the 
water until such time as the drone surveillance confirms no further 
sightings on the beach. 


The algorithm has a mean average precision (MAP) of 90.4% detec- 
tion, using true-positives and false-positives for the calculation. 
Precision here relates to how many correct images of sharks are 
identified from the total images (Exhibit 6.9). This is an impressive 
result. But it leaves open the question of false negatives—how many 
sharks aren't spotted (which seems important!)—but the ability to 
detect with high precision other species like dolphins is thought to 
minimize the number of false negatives. Like other deep-learning 
algorithms, the expectation is that results will improve further with 
additional data.'* In addition, trialing technology, such as multi- 
spectral cameras, is expected to provide better ocean penetration, 
particularly for cloudy days. 


"Communication with Paul Scully Power AM, CTO, The Ripper Company. 
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EXHIBIT 6.9 


The beach of the future, as The Ripper team pictures it, will incor- 
porate three technologies: 


1. A drone tethering system that allows the drone to be pow- 
ered and sit atop a cable with full view of the beach, able to 
operate 24/7, but most likely for the hours the beach has 
lifeguards on duty. 


2. Amachine learning algorithm linked to the drone video to pro- 
vide an alert that a shark is within proximity of surfers. 


3. Adrone payload called a shark shield® that can be dropped to a 
surfer close to a shark. The shark shield® emits electronic pulses 
that cause the shark to experience temporary muscle spasms 
and turn away from the electronic field. 


As you can see from these examples, machine learning works well 
when you want to produce a very accurate model or optimization 
path, and when you care less about understanding key variables 
that influence the decision-making process. This tends to be most 
useful when you already have an understanding of the decision- 
making mechanism, but it’s complex, and you have lots of diverse 
data examples. Machine learning neural networks can produce 
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impressive results by automating complex processes like human 
vision for pattern recognition. But many business decisions lack the 
sheer amounts of high-quality data required to employ learning 
algorithms.'? Some business and life decisions require subjective 
intuitions about past events and future issues unavailable in data, or 
involve guessing the responses of other players. Could you imagine 
delegating your decision to a machine about which city to move to, 
an algorithm that made the decision based on carefully weighing 
the preferences of thousands of other individuals and consider- 
ing your situation? Clearly not—though you might be interested in 
seeing the results. 


Crowdsourcing Algorithms 


Besides doing sophisticated analysis within your own team, we now 
have the ability to go outside the organization to harness more 
analytic power. Enter crowdsourcing as a way to pair enterprises 
searching for solutions with teams of individuals that have ideas 
for implementation. Participants often compete for a prize to solve 
challenging problems, which previously required the contracting of 
consultants and experts. Often problem solving competitions are 
presented at hackathons, daylong events where programmers and 
analysts compete for the best solution given the data available to 
solve the problem at hand. 


Kaggle is one such platform that hosts competitions. The range of 
prizes, challenges, and entrants is quite remarkable. Here are some 
recent ones." 


1. A challenge by the US Department of Homeland Security to 
generate a passenger-screening algorithm for a $1.5 million 
prize attracted 518 teams. 


'3Machine learning is as powerful and weak as the data used. If a model has sig- 
nificant errors—and we have seen examples of 10% or more errors in data sets—it 
inherently embeds all the biases of that data into its predictive efforts. 

“4Kaggle website. 
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2. Real estate company Zillow requested a model that would pre- 
dict home value, rewarding $1.2 million to the winning team 
from a field of 3779 teams. 


3. Predicting borrower default: 924 teams competed for a $5,000 
prize to assess data on 250,000 borrowers and come up with the 
best predictive algorithm for the probability that a person will 
experience financial distress in the next two years. 


4. Identifying fish species from cameras on fishing boats to esti- 
mate fish catch and fish stocks, in a contest format set up by 
The Nature Conservancy, where Rob is an Asia Pacific trustee. 
The prize of $150,000 attracted 293 teams around the world. 


5. Predicting survival on the Titanic using machine learning: Over 
10,000 teams entered the competition and were provided data 
on ship manifests, passengers, and survivors. 


The Good Judgment Project is an example of applying crowdsourc- 
ing to decision making at a large scale." Philip Tetlock, its founder, 
developed the project and website around the principle of the 
wisdom of the crowd. If enough people weigh in on a particular 
forecast, they'll cancel out any individual contributor’s biases and 
come to more accurate conclusions than a single forecaster would. 
This is a lot like when a crowd of people guess the number of mar- 
bles in a jar: A single guess may be very far from correct, but when 
enough people's guesses are pooled, deviations are canceled out 
and the crowd usually comes to the right conclusion. Forecasts 
were surprisingly accurate for major and sometimes very specific 
world events, including predicting military invasions. The best con- 
tributors are dubbed superforecasters, and the entire process is 
described in the book Superforecasting.'® 


Exhibit 6.10 shows a decision tree to help determine whether to 
use a crowdsourcing competition over other methods (such as an 
expert contract). It takes into account whether there is an achievable 


'SPhilip E. Tetlock and Don Gardner, Superforecasting: The Art and Science of Pre- 
diction (Crown Publishing, 2015). 
'STetlock and Gardner, Superforecasting. 
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goal, whether there are few or many resources available, and the 
willingness to take risks and incur costs. Not surprisingly, there 
have been entries on Kaggle, including finding a cure for malaria, 
with a large prize that wasn’t collected. You can see how it turned 
out not to be an achievable goal in the short term as few were in 
a position to believe they could win and the risks and costs were 
deemed to be large. 


WHEN TO CROWDSOURCE PROBLEM SOLVING 
USING COMPETITIONS AND PRIZES 
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EXHIBIT 6.10 


Source: Jonathan Bays, Tony Goland, and Joe Newsum, “Using Prizes to Spur 
Innovation,” McKinsey Quarterly, July 2009. 


Elon Musk’s Hyperloop project is an attempt to rapidly advance 
transportation technology. The Hyperloop is a pod transportation 
system currently under development, designed to transport peo- 
ple over long distances at astonishing, near supersonic speeds. 
Given its clear, well-defined goal, anda large, diverse population of 
problem solvers, many of whom are students willing to accept the 
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risk and hungry for the experience, the SpaceX Hyperloop competi- 
tion is a great example to test the power of crowdsourcing. With an 
available testing track at its disposal, SpaceX ran its Hyperloop Pod 
Competition in January 2017. By hosting a competition they were 
able to increase the quantity of prototypes being simultaneously 
explored. The jury is still out on Hyperloop as a viable concept, 
but the process is harnessing resources in a way not seen even a 
decade ago. 


Game Theory Thinking 


The analytic big guns we have discussed so far are based largely on 
statistical techniques. One of the other powerful tools in your tool- 
box for complex problems is game theory. Competitive situations 
in business and life mean that your actions influence and are influ- 
enced by another player, for these purposes called an adversary or 
opponent. For instance, your strategy has to consider your choices 
to compete aggressively or collaborate (where that is allowed under 
competition law). These strategic moves are sometimes played out 
over hours, days, or years. We employ game theory thinking to work 
through our own choices and competitor choices, set out in a logic 
tree. To assess how we should respond to an opponent's moves, 
we typically create a simulation in which we break the team into 
an attack team and a defense team, and make a series of moves 
that the other team has to respond to. We may cover as much as 
18 months of moves in a one-day workshop. Then we reconvene 
and have the business-unit leader reflect on the moves and coun- 
termoves, and the likely payoffs from each pursuing their best 
strategies. Game theorists use terms like minmax and maxmin to 
describe strategies where you choose an outcome that maximizes 
your minimum gain or conversely minimizes your maximum loss. 
These are constructs for formal games that go beyond the scope 
of what we are able to share in this book, which involve putting 
yourself in an opponent's shoes and crafting competitor responses. 
This is the right tool to use in difficult competitive and adversarial 
problem solving settings. 


We illustrate game-theory thinking with two different examples, an 
example from business (should you go to court?), and an individual 
one: Where should | serve in tennis? 
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Should We Go to Court? CSIRO Wifi Example 


If you ask a businessperson the question, “Should we go to court?” 
the answer is typically “no.” Going to court involves uncertain- 
ties about outcomes, lengthy periods of executive time in court, 
and substantial costs. But there are times when it makes sense, 
and game-theory thinking is a useful framework to help make this 
decision. Let's consider the case of a successful defense of a wifi 
patent by a small Australian government research organization 
called CSIRO against the world’s largest technology companies, 
including Microsoft, Intel, Dell, and HP. It’s a story that played out 
over more than decade in a series of passive and then increasingly 
aggressive moves by the parties. This strategy resulted in $400m of 
payments to CSIRO, which was used to fund additional research in 
the Australian national interest. 


CSIRO believed that its wifi technology was valuable, that its patent 
was being infringed, and that it was entitled to receive a reason- 
able fee for use. The giant tech companies who were using this 
technology, on the other hand, believed the right transfer price was 
zero for many reasons, including that CSIRO was a publicly funded 
research agency. In a CSIRO briefing to the US government, it was 
reported this way: “CSIRO tried to license this product. In 2003 and 
2004, it became aware of products that it believed infringed on its 
patent, and sent letters to 28 companies asking them to please dis- 
cuss terms for a license from CSIRO. The companies did not accept 
CSIRO's offer.” 17 


CSIRO had to decide whether to pursue its case in the courts or 
not. Doing so, it faced formidable companies in computing and 
networking with deep pockets and deep experience in patent 
fights. Besides the cost of legal action and uncertainty of a favora- 
ble outcome, CSIRO faced potential costs from being countersued 
by the tech companies. On the other hand, not defending its pat- 
ent set a precedent for any future licensing deals it might seek to 
do in the United States. 


CSIRO briefing to US government, December 5, 2006, https://wikileaks.org/ 
plusd/cables/07 CANBERRA1721_a.html. 
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Here's how they reached the decision to go to court. The execu- 
tive director of Business Development and Commercialization 
for CSIRO at the time of the decision, Mehrdad Baghai, a former 
colleague of Charles and Rob, put it this way: “Going back to the 
beginning, we first of all had to satisfy ourselves that we had a claim 
that we could pursue. That involved spending about $1 million to 
get some preliminary expert advice. Then we estimated what we 
could expect to receive if our claim succeeded in court. | felt it was 
upwards of $100 million and possibly over $1 billion. To get there 
meant we needed to commit $10 million to the initial court action. 
As a back of the envelope calculation, we needed a 10% probabil- 
ity of winning. | felt we had that, easily. | also felt we had a fallback 
option in the event of costs blowing out, that was an abandonment 
option, the option to sell all or part of the claim to a group that 
specializes in intellectual property litigation.”'® Having got to the 
point of being prepared to go to court, here’s how they used game 
theory thinking to win. 


In competitive strategy for business, the cleaving lines are often 
around where and how to compete. Strategies are often depicted 
contrasting competitor choices about market segments, and 
whether they compete on cost or value. CSIRO’s defense of its 
patent for wifi provides an example of how strategy plays out over 
time, in a series of moves designed to learn more about competi- 
tor intentions, and to position their own moves for the highest level 
of success. 


CSIRO's strategy for how to compete went from an unsuccessful 
passive approach of letter writing to ask for voluntary licensing of 
its technologies, to suing the big tech giants. Even more dramatic 
was their strategy about where to compete. CSIRO had to decide 
which players it should target initially (Exhibit 6.11). It chose to sue 
a small to medium networking company called Buffalo Technology 
for patent infringement in what has been described as a test case. 
Buffalo was a Japanese company rather than a US giant, with less 
experience in US patent cases, and at that time it had an almost 
complete reliance on networking technology that was transitioning 


Private communication with Mehrdad Baghai. 
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DAVID VERSUS GOLIATH 
WIFI PATENT INFRINGEMENT 
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EXHIBIT 6.11 


rapidly to wifi. CSIRO’s advisers reviewed the record of patent liti- 
gation in the US to assess their chances of success. They noted 
that the Federal Court for the Eastern District of Texas had a much 
higher success rate for patent holders, and that the time to court for 
plaintiffs was much faster than for other jurisdictions. For the period 
1995-2014, which includes the wifi case, the success rate for plain- 
tiffs in Texas Eastern was 55% compared to 33% overall. They also 
took note of the fact that trial success rates for plaintiffs with juries 
were higher than for the judicial bench; for the period 2005-2009 
they were 77% compared to 58% for the bench." 


Following CSIRO lodging the suit against Buffalo Technology, coun- 
tersuits were lodged by other patent infringers, including Intel, HP, 


“PriceWaterhouse Coopers, Patent Litigation Study: A Change in Patentee 
Fortunes, 2015. 
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Microsoft and Dell. CSIRO had to decide what to do in response 
to this aggressive action. CSIRO responded the following year 
by adding another eight companies to its infringement list. Soon 
CSIRO was facing 14 companies in a jury trial. Their nerve lasted 
and HP settled first, followed by the 13 others for $205 million. The 
action CSIRO brought in 2009 against ATT, Verizon, and T Mobile 
was settled before trial in 2012 for an additional $220 million, a year 
before the patent was due to expire! 


Decisions to go to court require a problem solving approach that 
takes into account how an adversary will respond. In this case 
CSIRO was smart to pick a weak adversary in a favorable court to 
demonstrate its patent, and then it chose to fight larger players 
from a stronger position. Furthermore, CSIRO had decided they 
could afford to lose $10 million to put themselves in the game. 


Where to Serve in Tennis: Game Theory Example 


Many of us play tennis and, as a result, engage in competitive strat- 
egy. Barry Nalebuff, a game theorist at Yale Management School, 
has analyzed where to serve in tennis with fellow strategist Avinash 
Dixit. Their answers are simple and powerful. The recommenda- 
tions are to remove predictability by making each serve appear 
random, and to place your serves to your opponent's forehand 40% 
of the time and to their backhand 60% of the time. The analysis 
was described for two types of serve, to the opponent's forehand, 
and to their backhand, and where the receiver anticipated one 
serve or the other. The likelihood that your opponent success- 
fully returns serve is highest if you serve to their forehand and he 
or she anticipates this by moving to the right place. Nalebuff and 
Dixit prepared a payoff table to reach this conclusion that we show 
below (Exhibit 6.12). You read it as follows: 


e |f you serve to your opponent's forehand and they anticipate it, 
they will return the serve 90% of the time. 


e If you serve to their forehand and they do not anticipate it, they 
will only return successfully 30% of the time. 


2Avinash K. Dixit and Barry J. Nalebuff, Thinking Strategically (W.W. Norton, 1991). 
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Source: Avinash K. Dixit and Barry J. Nalebuff, Thinking Strategically (W.W. 
Norton, 1991). 


e If you serve to their backhand and your opponent anticipates it, 
they will return the serve 60% of the time (assuming their back- 
hand side is weaker, as it is for most amateur players). 


e If you serve to their backhand and they do not anticipate it, they 
will only return 20% of the time. 


Rob took these findings to heart and tried to apply them to his own 
game, where he is an amateur lefthander who plays mostly on grass 
or synthetic grass. 
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Since Nalebuff and Dixit’s Thinking Strategically was published in 
1991, there has been a big change in tennis racket technology and 
as a result, serving power. Commentators now talk about three 
types of serve: wide, to the body, and down the T (see Exhibit 6.13). 
When you watch top players receiving serve today they position 
themselves in a 45-degree line to the server and don’t do much 
anticipating, just find a position that gives the best chance of return- 
ing aserve from all three places. Does this invalidate Nalebuff and 
Dixit'’s conclusion that the server needs genuine unpredictability 
of each serve, and that the serves should be in a 40/60 mix? Rob 
wanted to know. 


Body Down the T 
Wide 


Server 


EXHIBIT 6.13 


Data analytics comes in to help us with this question. Since the use 
of Hawkeye technology for line calls, we can analyze what serv- 
ers actually do in serving and measure predictability. We can also 
link it to what analysts call important points or clutch points—that 
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is at 30-40 and Advantage out, which are always served to the Ad 
court. GameSet MAP reviewed the 2012 Olympics final between 
Andy Murray and Roger Federer using GIS technology. This was 
a match Murray won. They noted that Murray served with more 
spatial variation (less predictability) on clutch points than Federer, 
and that he directed seven of eight serves wide to Federer's 
backhand.” That doesn’t sound unpredictable, but he was serv- 
ing to Federer's backhand, his weaker side. The unpredictability 
came from mixing one serve in another direction to sow a seed 
of doubt. On clutch points left handers will serve to the Ad court 
predominantly because they have the advantage in serving the 
big carving wide serve to the backhand of a right hander. It’s a 
weapon, so you use it. Rafael Nadal will go wide probably two- 
thirds of the time on the advantage side, but mix up body and 
T-serves with the rest. 


Game-theory problems can be set out as decision trees as well as 
payoff matrices. We have put the factors described above into a 
tree to help Rob think about what he should do in deciding where 
to serve. We start with whether the server is left or right handed, 
whether the opponent is left or right handed, whether the oppo- 
nent is stronger on the forehand than backhand and whether the 
serve is to the Deuce court or the Ad court. We stopped the tree 
there as it captures the essence of the game. (We could have added 
two other branches for completeness: whether it is a clutch point 
or a regular point, and whether it is a first or second serve). The mix 
that results is shown in Exhibit 6.14. We show only two cases: a right 
hander serving to a right hander, like Federer versus Murray, and 
a left hander serving to a right hander, like Nadal versus Federer. 
The percentages are in effect conditional probabilities. They show 
considerable variation depending on the opponent and whether 
you're serving on the deuce court or ad court. This analysis con- 
cludes that it is no longer a simple 40/60 mix as per the two-serve 
place example, but one with a richer mix. 


1GameSet Map, February 19, 2013. 
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WHERE TO SERVE THE BALL DURING A TENNIS GAME 
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So the tennis greats are lined up at the baseline ready to serve with 
their own algorithm based on these factors, plus the importance of 
the point, and whether it's first or second serve. The conclusion of 
Thinking Strategically, to make each serve appear random remains 
a sound one, albeit with lesser unpredictably to take advantage 
of your strengths and opponents weaknesses. This is the key aim 
of every game theorist. The conclusion we reach about serve 
direction is conditional on three to five questions. When you're 
standing at the service line, you have to quickly take these factors 
into consideration. The result is a mixing strategy as the authors 
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of Thinking Strategically argued, but clear preference in the left- 
hander's case for the backhand side on clutch points and first serves. 
Take the tree and place yourself on it and figure out how you should 
serve on important points to a regular opponent in singles. 


Complex problems often require the big analysis guns we have 
described. We have examined the sophisticated tools now avail- 
able to problem solvers. We expect them to be employed more 
in the future as more large data sets become available, as experi- 
ments become cheaper, and as software packages and scripting 
languages become more widely used. Remember to start your 
analysis with time-tested heuristic short cuts, simple statistics, 
and root-cause analysis before taking on sophisticated tools. This 
initial step may lead you to the decision that these more sophis- 
ticated tools aren't needed for your problem situation, and will 
in any case give you important information on advanced tools 
to employ. 


Takeaways 


e Getting to a solution for many complex problems means you 
will need to be aware of and know how to use sophisticated 
analytic tools. 


e The starting point is always to get a thorough understanding of 
your data through graphing, visualization, and summary statistics. 


e Which advanced tool you use is often dictated by whether you 
are seeking to understand drivers and develop an intervention 
strategy, or predict outcomes and plan accordingly. 


e Experiments are a powerful and often overlooked tool in the big 
gun arsenal; if you can't make one, sometimes you can find a 
natural experiment. 


e Machine learning is emerging as a powerful tool in many problem 
spheres; we argue to understand problem structure and develop 
hypotheses before employing deep learning algorithms (huge 
mistakes can come from bad data and bad structuring, and these 
models offer little transparency). 
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e You can outsource problem solving, including deep learning, 
through crowdsourcing on platforms such as Kaggle. 

e Where there is an adversary whose behavior many change in reac- 
tion to your choices, you can look to game theory approaches 
with logic trees to work out the best course of action. 


Problems to Try on Your Own 


1. What big gun would you select for understanding causes of 
mental-health problems using the framework in Exhibit 6.1? 


2. Set out how you would predict Olympic games medals for the 
top five countries at the next Olympics. 


3. How would you test effectiveness of social media campaigns 
using experiments in your business or nonprofit? 


4. How would you set out the logic tree to decide whether to take 
a competitor to court where you believe they have appropri- 
ated your IP? 


Chapter Seven 


Synthesize Results and 
Tell a Great Story 
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Introduction 


The final elements in the seven steps process, synthesizing your 
findings and telling a compelling story, can be the most fun. It is the 
culmination of all your hard work, and, with luck, answers the difficult 
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question you set out to solve. Done right, your conclusions are an 
engaging story, supported with facts, analyses, and arguments that 
convince your audience of the merits of your recommended path. 
This can attract volunteers to your social causes, funding for your 
social organization or start-up company, build harmony in your fam- 
ily, or boost confidence in your own career path. Good problem 
solving and the story telling that goes along with it make our lives 
better—for problem solving done well translates into action that 
improves our circumstances. 


Of course it doesn’t always work out like this. Most readers will have 
experienced a project at work or in a volunteer organization that 
got through the analysis stage, and then fizzled. Failures at the later 
stages of problem solving have a number of causes: where giant 
models are not quite finished, reams of data produced without a 
clear point of view, conflicting interview or market survey data, or 
team members at odds over the right way forward. As young con- 
sultants, we dreaded the nervous presentation of detailed results 
that never jelled into a real story; in McKinsey this was called the 
anxious parade of knowledge, or APK. 


If you follow the straightforward seven-steps process we have 
described, you should never succumb to deadlocked teams, con- 
fused findings, or empty APK. You will have been driving your 
logic trees from component disaggregation to strong and testable 
hypotheses, you will have followed good team processes to surface 
alternative hypotheses early on and avoid confirmation bias and 
other problem solving mistakes, and you will have practiced one- 
day answers that summarize your best understanding at any point 
in the process, supported with synthesis of your best evidence at 
that juncture. 


This chapter addresses the transition from one-day answers to more 
complete story telling. It addresses approaches to synthesis of data 
and analysis, using pyramid structuring of your overall case, induc- 
tive and deductive support for your argument, and compelling 
story telling, including to challenging audiences. 


Synthesis of Findings 


Synthesis of Findings 


The quality of your synthesis will determine the quality of your 
decision making. This is why it pays to triangulate your views with 
people who you know synthesize well. 


—Ray Dalio’ 


The first step toward compelling story telling is to synthesize the 
findings from your data gathering, interviews, analysis, and mod- 
eling. If you have been following the seven-steps approach we 
outlined in Chapter 1, this should be relatively straightforward. 
Charles learned the power of this approach on his first McKinsey 
project. He got on a plane to Calgary for the first meeting with the 
client and found himself sitting next to the senior partner on the 
project, who nodded a greeting and went back to his work. He was 
furiously scribbling on a pad of paper and ripping off the sheets. 
When Charles worked up the courage to ask what he was doing, 
his colleague answered, “| am writing the final presentation.” Final 
presentation? How could that be? They hadn't even met the client. 
He answered, “Look, | know my hypotheses are early stage and 
some will be wrong, but if | guess at the answers now, l'Il know 
what questions to ask the client when we meet their team.” And he 
was right. Imagining a set of hypotheses allowed the team to begin 
focused data gathering from the first meeting. 


Our recommended process is iterative at each stage and driven by 
the interaction of the strong hypotheses of your one-day answers 
with the analyses of your workplan. Alternative hypotheses should 
already have been tested and embraced or rejected, and dead-end 
analyses pruned off. Regular team meetings (even if you have to 
press family and neighbors into service in makeshift teams) should 
have already begun to link analysis and emerging findings in a 
convincing way. 


‘Ray Dalio, Principles: Life and Work (Simon & Schuster, 2017) (referenced in Shane 
Parrish’s blog Brain Food #233). 
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As you move to final synthesis, you draw together the individual 
findings of the work on each branch and twig of your logic tree into 
an overall picture. Humans are mostly visual learners, so it is often 
useful to represent each of your findings in the form of pictures or 
graphics that highlight the insights that emerged from your work. 
Where possible, the most powerful visualization is to show each 
graphic as branches on your revised tree structure. Exhibit 7.1 is 
an example from the battle of the hardware company giants we 
introduced in Chapter 1. 


This kind of visualization of findings illuminates the overall problem. 
It allows for insights to emerge that cut across and combine the 
different branches of analysis. We can see the overall story begin to 
emerge from individual branches on revenue, costs, and asset pro- 
ductivity. Looking at the relative sales growth curves, we know that 
one business (Home Depot) far outstripped the other (Hechinger); 
examining the selling price data, purchasing costs, logistics 
expenses, overheads, store design, and inventory turns starts to 
paint a fuller picture of what drove the different revenue growth 
results. Lower prices drove higher sales growth; sales volume, along 
with better store design and logistics systems, allowed higher asset 
and inventory turnover and lower costs per unit sold in every part of 
the business. The synthesis in pictures in Exhibit 7.1 begins to tell a 
visual narrative of competing business models. 


We encourage you to draw pictures from the analysis that synthe- 
size your findings. There are good primers on how to do this from 
the methods we learned at McKinsey such as Gene Zelazny’s Say It 
with Charts to Cole Nussbaumer Knaflic’s Storytelling with Data.? 
As a young consultant in New York, Rob would take his pack of find- 
ings to Gene Zelazny, only to have Gene say, “Now what are you 
saying here Rob? These charts don't really support what you just 
told me you've concluded so let's ditch them. I'm really interested 
in that point you made about declining market share of the client 


2Gene Zelazny, Say It with Charts: The Executive's Guide to Visual Communications 
(McGraw-Hill, 2001), and Cole Nussbaumer Knaflic, Storytelling with Data: A Data 
Visualization Guide for Business Professionals (Wiley, 2015). 
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RETURN ON INVESTED CAPITAL ANALYSIS 
HECHINGER VS. HOME DEPOT 
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but you don’t have a chart on that. Lets get that in the pack to 
support your findings.” This sometimes bruising process, despite 
Gene's positive and encouraging style, became the boot camp for 
consultants to learn how to synthesize findings and support them 
with graphics. 


Good team process is important here as in the other steps. We like 
to bring the whole team together for synthesis sessions and have 
each team member present their analyses and results to the assem- 
bled group. This gives another checkpoint to pressure test alterna- 
tive hypotheses, and it provides the foundation for moving from the 
simple situation-observation-resolution story summary to a more 
complete and supported argument. 


This is often the stage where a big insight emerges that wasn’t 
obvious in the branch-level analysis. In our BHP mining example in 
Chapter 8 the analysis of the viability of the single mineral deposit 
absorbed the team right up to the point that synthesis showed the 
huge potential value in other nearby deposits that were unlocked 
by the transportation infrastructure required to develop the first 
mineral deposit. 


From One-Day Answers to Pyramid Structure 


As we move from the early days of problem definition to work plan- 
ning and completing analyses, our reasoning is sharpened. Here is 
a graphical way of showing this (Exhibit 7.2). 


In the early stages, our porpoising into data with well-designed 
analyses is a process of reasoning toward implications. We hypoth- 
esize a one-day answer using a situation-observation-resolution 
structure and we constantly pressure test this with our analyses 
under the structure of our workplan and team processes. As we 
discussed in Chapter 3, these one-day answers are critical to the 
process. Summarizing the situation allows us to update our best 
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understanding of the problem; observation or complication pro- 
vides the tension in the problem, what isn't working, and our best 
insights into ways to unpick it; resolution is your best understanding 
of the solution path that moves us toward the answer. 


As the workplan is executed and our critical path analyses begin to 
yield fruit, our reasoning mode shifts from understanding toward 
marshaling reasons to drive action. We move from simply stating 
the implications of analyses toward how those results can motivate 
a plan for change. The seven steps approach is always oriented 
toward evidence-based action—it aims to answer “What should we 
do and how should we do it”? 


Telling Compelling Stories 
Once you have synthesized your findings into a series of pressure- 


tested and convincing graphics, you can move to the final step 
of structuring a compelling communication for your audience. 
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The starting point for this is to return to your problem definition 
worksheet and remind yourself: 


e What problem are we trying to solve? How has this evolved? 

e What are the key criteria for success that our decision maker 
(which may be yourself) set out in advance? It is important to 
reflect these explicitly in your story. 

e Did you honor the boundaries of the problem set by the decision 
maker? If not—which may be for good reasons around creativity 
or deciding to relax a constraint to open up new possibilities— 
you need to make the case here. 


The visual structure we use to represent our story structure is tradi- 
tionally a pyramid—which is really just one of our logic trees turned 
on its side (Exhibit 7.3). This is the standard organizing structure 
taught to journalists and consultants, and was popularized in the 
problem solving literature by our former McKinsey colleague Bar- 
bara Minto.’ This visual helps show clearly how each element of our 
argument is supported by data, interviews, and analyses. 


BASIC PYRAMID STRUCTURE 


Governing thought— 
your answer to the 
audiences question in a 
[| single statement 


Situation 
Observation 
Resolution 


Key line—the 
core logic of 
your case 


Support—on 
which your 
logic relies 


EXHIBIT 7.3 


3Barbara Minto, The Pyramid Principle, 3rd ed. (Prentice Hall, 2009). 
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At the very top level is our lead or governing statement of the 
problem. Not surprisingly, your latest situation—-observation— 
resolution statement is usually the best governing statement. 
Depending on the audience, you can vary the order of these 
three statement elements. As we will show below, sometimes it 
is best to carefully lead the audience from situation to observa- 
tion to resolution, which are your recommended actions. But our 
bias in most circumstances is to lead by answering the question 
“What Should | Do?” and then summarize the situation and key 
observations that support action. 


Using the insights from your synthesis stage, you then begin to fill 
in supporting arguments that back up your top-level answer. There 
are a number of different ways to do this depending on the nature 
of your answer and your audience. Exhibit 7.4 shows two structures 
that we often use. 


STRUCTURING YOUR ARGUMENTS 
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As you have likely already recognized, our basic deductive logic 
(from general principles to individual cases) and inductive logic 
(from individual observation to general conclusions) approaches 
to generating answers introduced back in Chapter 3 underpin 
these two structures (Exhibit 7.5). You can use both inductive and 
deductive structures in the same tree. 


ARGUMENT TYPES 


DEDUCTIVE INDUCTIVE 


Argument Effect or 
summary Inference 


General 
principle 


Actual | 
active ideas or Situationa 
ideas 


Specific Cond 
— > [observation || -0ncusion | cone | Cause Cause 


EXHIBIT 7.5 


There is no general preference for inductive or deductive reasoning 
in your pyramid structure, though Minto argues that constant refer- 
ence to deductive arguments can feel labored or pedantic. 


To complete this section, let's look at the complete narrative of 
our battle of the hardware companies case in Exhibit 7.6. This 
draws together the evidence from our analysis phase into a syn- 
thesis of the findings, and then tells the story: Hechinger needed 
to change its business model quickly to address the competitive 
threat of Home Depot. 
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HECHINGER DRAFT STORYLINE 


RESOLUTION: Hechinger must work to develop and implement a lower cast, higher volume 
hardware retail store model to compete in markets which Home Depot is in or 
planning to enter 


SITUATION: A new warehouse superstore model has emerged and is planning expansion 
concurrent with Hechinger'’s expansion and into Hechinger’s markets 
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revenue very quickly 
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Here you can see the whole story on one page, with the govern- 
ing thought and call to action at the top (provided by our concise 
situation—observation-resolution structure), the three major argu- 
ments made that underpin the governing thought, and supporting 
arguments and data that provide the proof for the need for action 
and the formula for change. The next stage from this storyline 
summary is to storyboard out your entire presentation. We usu- 
ally divide our paper into nine boxes and cartoon or dummy each 
exhibit, with the storyline leads across the top of each cell. This is 
what Cole Nussbaumer Knaflic calls developing and checking the 
horizontal logic of the narrative.* Humans are storytelling creatures, 
not logical robots. Remember that as you craft your messages. 


How to Deal with Difficult Audiences 


A number of years ago Rob and Charles were working with a refinery 
company in a remote island location, owned by a larger company 
based in Australia. It was a difficult situation. The local manage- 
ment team resented the foreign owner meddling in their business, 
and they were resistant to change. The consulting team worked as 
best we could, employing the problem solving process we have 
introduced here, and came up with a final answer that looked like 
Exhibit 7.7. 


As you can see, the recommendation was for the refinery business 
to cut costs substantially and become a modest growth, niche oper- 
ator. While this conclusion, based on comparative analysis of com- 
petitors, was rock solid, it was not one that the local management 
team wanted to hear. In circumstances like this, it can make sense to 
use a revealed approach to your arguments, employing a decision 
tree structure rather than the traditional pyramid. Exhibit 7.8 is an 
example of what that might look like. 


With a decision tree final storyline structure, you provide evi- 
dence for each yes/no branch in your tree, slowly working the 


“Cole Nussbaumer Knaflic, Storytelling with Data (Wiley, 2015), 181. 
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OILCO STORYLINE 
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niche market by cutting the 
rate of growth of costs by x 
percent a year 


lt is not viable to 
continue to run 


The local market 
will remain an 


A review of expenditure 
shows Oilco can cut 


the Oilco refinery aa attractive niche =$——7 tg æst growth by x% 
as a platform for market for low- and so restore margin 
growth cost operators growth to y% 
Competitors New market The costs The market Good margins Reduce 
are better entry requires of Oilco of Is expected can be capital Streamline | Cut 
positioned different acquiring to generate achieved by eypenditure OPEFAMONS) Wacol POrgke 
than Oilco capabilities comparable moderate operators P overhead 
to capture than Oilco capabilities revenue growth who can keep 
growth niche are not with limited their costs 
opportunities justified competition down 


EXHIBIT 7.7 


decision maker toward your conclusion. You reveal the answer 
rather than leading with it. In the Oilco case the team carefully 
revealed compelling competitor data, layer by layer, that helped 
the local management team get comfortable with the difficult 
conclusions. 


In summary, the final stages of good problem solving are to syn- 
thesize the findings of your analysis in a way that highlights your 
insights. You are then prepared to revisit your original problem 
definition and answer your decision maker's question, “What 
should | do?” in compelling way that motivates action. The pyr- 
amid structure helps to structure arguments and support into a 
powerful story. 


Synthesize Results and Tell a Great Story 


OILCO DECISION TREE ARGUMENT STRU 
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Chapter 7 Takeaways 


limit standalone 
downside 


Pursue low-cost local 


operator approach 


Secure outside equity 
partnerlexit 


e Synthesis brings together all the separate pieces of your analytic 
work and often yields new insights you didn’t notice when you 


were in the weeds of analysis. 


e To avoid the dreaded anxious parade of knowledge, use your 
logic tree structure pyramid to organize a compelling story. 


e Your one-day answer 
resolution, kept up to date for your findings, is 


structure of situation—observation— 


usually the start- 


ing place for the governing thought of your narrative. 
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e Try several storyline structures to see which are most clear and 
compelling; sometimes a decision tree format that reveals the 
answer step-by-step is useful with difficult conclusions and 
audiences. 


e Use a storyboard to plan out your deck of slides and have your 
leads be the argument of the story (not boring titles); read across 
the slides to check horizontal logic and coherency. 


Problems to Try on Your Own 


1. Doastoryline pyramid for Rob's solar panel problem in Chapter 1. 


2. Take any good newspaper story and put the arguments in pyra- 
mid structure. 


3. Look at Lincoln’s Gettysburg address or Marc Antony's speech 
in Shakespeare's play Julius Caesar—what narrative structure do 
they use to be so compelling? 


Chapter Eight 


Problem Solving with 
Long Time Frames 
and High Uncertainty 


| THE Future... | 


| ; | 
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As problem solvers, we shuffle through a deck of strategies when we 
face high uncertainty. We edge into risk, looking for the opportuni- 
ties to learn before acting precipitously, hedging our bets where we 
can, and taking steps that build capabilities and future resilience. 
We know there is value in being systematic and working through a 
range of potential strategies when faced with uncertainty and long 
time frames. 


As we have seen, the seven-steps framework is robust and adapt- 
able in addressing a wide range of personal, work, and societal 
problems. Many of the examples to date exhibit low to moderate 
uncertainty, where the cost of error is relatively low, and where time 
frames are relatively short. But there is a class of problems where 
time periods are long, complexity and uncertainty are high, and the 
consequences of error are substantial. The case of protecting Pacific 
salmon that we have looked at in previous chapters is a good exam- 
ple. Outcomes for salmon depend on many variables, including 
inherently uncertain ones like climate change. Species success plays 
out over decades rather than months, and the consequences of bad 
outcomes are broadly felt in human incomes and ecosystem losses. 
We believe the seven-steps model is useful for complex problems 
with high uncertainty, particularly the emphasis on an iterative 
approach, but we will add some frameworks to our problem solving 
kit to manage these more difficult, long-term problems, some of 
which involve putting the analytic tools of Chapter 6 into practice. 


In this chapter we define levels of uncertainty and the implica- 
tions for problem solving. We provide a problem solving toolkit 
with escalating actions to address different levels of uncertainty. 
We demonstrate the tools via a number of cases where we have 
employed these approaches, from long-term capital investment in 
resources, to making career choices, to determining whether you 
risk outliving your savings, and to developing strategies comprising 
a portfolio of initiatives. 


Levels of Uncertainty 


There are many ways to understand and evaluate uncertainty, includ- 
ing statistical measures of variability and conditional probability. 
Previously, we lacked a way to describe and respond to different 


Levels of Uncertainty 


levels of uncertainty, without having to make spot predictions that 
could dangerously underestimate the range of uncertainty. Several 
of our former McKinsey colleagues addressed this need with a five- 
level schema for assessing uncertainty.' They brought new language 
into play in tailoring actions to uncertainty, such as “big bets” and 
“no regrets” moves that are now in widespread use. In this model, 
uncertainty levels range from level 1, what is a reasonably predict- 
able future, to level 5, the unknown unknowns (Exhibit 8.1). Level 1 
uncertainty, such as simple predictions and short-term forecasts, 
is discussed in earlier chapters and can be addressed with good 
team processes. More challenging settings are where the level of 
uncertainty rises to levels 2, 3, and 4. 


WHAT IS THE LEVEL OF UNCERTAINTY? 


UNCERTAINTY LEVEL HOW UNCERTAINTY IS DEFINED EXAMPLE 

Unknown unknowns 5 Unexpected or unforeseen conditions eteorite hits Earth 
4 True ambiguity—impossible to predict anhattan sea level 2050 
3 A range of futures Energy balance 2025 


2 Alternative futures rexit 


Known unknowns | Reasonably predictable futures obile phone sales 


EXHIBIT 8.1 


e Level 2 are situations in which there are alternative futures that 
could follow from a legislative change or technology change, 
where the results will be binary in the sense of who wins/loses. The 
Brexit decision in the UK is an example. Another is how should 
auto insurers prepare for driverless cars on the road, where the 
time frame for adoption is uncertain, and the role of carmakers 
in providing the insurance is unclear. Both of these examples are 
likely to have uncertainty resolved in a 5- to 10-year time period. 


‘Hugh Courtney et al., “Strategy Under Uncertainty,” McKinsey Quarterly, 2000. 
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e Level 3 is where there is a range of futures that are possible 
but where it isn’t clear which one is more likely. Where fossil 
fuels fit in the energy mix in 15 years is an example. Scenarios 
range from similar to today all the way to a very diminished role. 
Another frequently discussed topic is what jobs will we have 
in the future with advances in artificial intelligence/machine 
learning and robotics. This will be clearer in a 10- to 20-year 
time frame. 


e Level 4 describes true ambiguity, where it is not possible to con- 
ceive of all the potential outcomes or to predict them with confi- 
dence. You could put the issue of sea levels in Manhattan in 2050 
into that category, although some climate scientists will have 
different views. In an update of “Strategy Under Uncertainty,” 
author Hugh Courtney gave the example of early-stage biotech 
investments having Level 4 uncertainty.” 


e Level 5 is the nearly absolute uncertainty of events that are 
unpredictable with current knowledge and technology, some- 
times called unknown unknowns. 


Uncertainty levels 1 through 4 are colorfully called “the known 
unknowns,” in contrast to level 5. We don’t put them in a too-hard 
basket. Instead we approach them recognizing and quantifying 
the level and type of uncertainty, and then develop approaches to 
move toward our desired outcomes by managing the levers that we 
control. The next and harder step is to identify what actions you can 
take to deal with a particular level of uncertainty. 


It is important to remember this as we dive in to this topic: Uncer- 
tainty can be a good thing for strategic problem solvers! Hedge 
fund and other clever investors hope for uncertain and volatile mar- 
kets—provided they have an analytic edge. If your problem solving 
is right you can earn good returns and guard your downside while 
others are floundering. 


?Hugh Courtney, “A Fresh Look at Strategy Under Uncertainty: An Interview,” 
McKinsey Quarterly, December 2008. 
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Logic Trees to Deal with Uncertainty 


The spectrum of actions you can take when faced with level 2 to 
level 4 uncertainty range from buying time (which you can also call 
do nothing, wait for more information), all the way through to tak- 
ing big bets and no regrets moves (Exhibit 8.2). You will typically see 
actions linked to the level of uncertainty as follows. 


e Buying information, which may involve data gathering and 
analysis of underlying processes, to clarifying the sources of 
uncertainty and the motives of protagonists. The information 
for level 3 and 4 uncertainty may be modeling for distant events 


WHAT YOU CAN DO TO ADDRESS UNCERTAINTY 


Do nothing/buy time 

Buy information 

Buy hedges 
ACTIONS Buy insurance 

Buy low-cost options 

No regret moves 


Big bets 


EXHIBIT 8.2 

Source: Albert Madansky, “Uncertainty,” in Systems Analysis and Policy Planning, 
edited by Edward S. Quade and Wayne |. Boucher (American Elsevier Publishing, 
1968); and Hugh Courtney, Jane Kirkland, and S. Patrick Viguerie, “Strategy 
Under Uncertainty,” McKinsey Quarterly, June 2000. 


3Uncertainty, Chapter 5 by Albert Madansky in Systems Analysis, Quade and 
Boucher, RAND Corporation, 1968. 
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rather than fact gathering of observable things. There is likely to 
be a high cost of moving toward perfect information and a point 
at which it is too costly to acquire more information.‘ 


e Hedges are a way of dealing with uncertainty that involves 
making a reasonable cost move or investment that will counter 
downside events. An example is fossil fuel companies investing 
in renewable energy. Other examples of hedges involve buy- 
ing water rights in areas expected to have lower rainfall with 
climate change. Sometimes these can involve options, futures 
contracts, or other financial instruments—although few finan- 
cial hedging mechanisms are as long-lasting as many difficult 
problems require. 


e Acquiring low-cost strategic options through a portfolio of ini- 
tiatives, or betting on multiple horses in the race, is another way 
we deal with uncertainty. Major financial institutions are investing 
relatively small equity stakes in financial technology (fintech) start- 
up companies to be in a position where they can first understand 
and then capitalize on innovation that challenges established 
players. We saw how important information gathering is in times 
of disruptive technologies in our newspaper versus new media 
example in Chapter 2. These are typically low-cost options, and 
often involve bets on competing technologies. How IBM lost the 
PC business is put down to a failure to exercise the low-cost options 
itheld in its key suppliers. “Early on IBM had the financial resources 
to take a large equity position in both Intel and Microsoft. It did 
make an investment in Intel in 1982, getting a 20% equity stake 
and warrants to purchase another 10%. But, ironically, IBM sold out 
in 1986 and 1987 for $625 million. Ten years later that 20% stake 
would have been worth $25 billion. As for Microsoft, IBM had an 
opportunity to buy 30% of Microsoft for under $300 million in 
mid-1986. By the end of 1996 that $300 million investment would 
have been worth $33 billion.” That is just the financial investment 
loss—the loss in learning value is incalculable. 


“James C. Morgan and Joan O'C. Hamilton, Applied Wisdom: Bad News Is 
Good News and Other Insights That Can Help Any Anyone Be a Better Manager 
(Chandler Jordan Publishing, November 2016). 

‘Adam Brandenberger and Barry Nalebuff, Co-Opetition (Currency Doubleday, 
1996), 156. 
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e Buying insurance. The threat of global warming is sometimes 
framed as an insurance problem, with estimates that a little over 
1% of global GDP has to be committed to paying the annual pre- 
mium to keep CO, levels below a 2-degree centigrade increase. 
That hasn't happened to date but there is some progress toward 
international agreements that would move toward this. There 
are also insurance products for a range of uncertain situations, 
known as cat bonds or catastrophe bonds, that cover events like 
typhoons or hurricanes. 


e No regrets moves come into play when you are comfortable 
with the level of uncertainty, or can edge out into a competitive 
space with incremental learning moves; this often involves the 
capability building you'll need no matter what the outcome. 


e Big bets are taken when you have a level of confidence about 
an outcome not shared by others. The Big Short book and film, 
describing the investments in shorting subprime mortgage credit 
default swaps in the US, is a great example of a big bet when 
uncertainty remained high. The investors spotlighted in the 
book felt the mortgage securities were mispriced and took big 
bets when others weren't prepared to. Of course, big bets can 
also go bad when uncertainty resolves itself in ways that leave 
an asset investment stranded. For example the generous feed 
in tariffs for Spanish solar investment a decade ago looked to be 
too good to be true. In this case the assumptions made about 
the behavior of host governments turned out to be incorrect. 
Where uncertainty sits at level 2 or greater, most decision makers 
will seek to defer irreversible commitments. That can work in the 
absence of competition but becomes a more difficult equation 
where there is the possibility of preemption. Big bets are open 
only to a few, as Hugh Courtney pointed out—those with busi- 
ness models that generate a lot of cash and have low debt. They 
can be shaping winning strategies, as a recent book by some of 
our former colleagues demonstrates.’ 


éMichael Lewis, The Big Short (W. W. Norton, March 2010). 
’Chris Bradley, Martin Hirt, and Sven Smit, Strategy Beyond the Hockey Stick: 
People, Probabilities, and Big Moves to Beat the Odds (Wiley, 2018). 
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When we can estimate the parameters of uncertainty, mathematics 
can help us calculate what is a fair bet and estimate the value of dif- 
ferent options. We can estimate conditional probabilities through 
experiments and sampling, and we can use joint probabilities to 
estimate complex risks involved in a decision. All this requires 
thinking probabilistically, a term a statistics colleague of Rob's 
uses. In her view, our inherent biases in decision making make us 
reasonably bad at this without systems and good mathematical dis- 
cipline. She calls humans “the unintuitive statistician.”® As we know 
from Chapter 5, we can fight these biases with good processes and 
team norms. 


Case Examples of Long Time Frames 
and Uncertainty 


Now that we have a framework for assessing uncertainty levels and 
some actions for addressing risky situations, let’s look at a range 
of individual, business, and societal level cases to see how strat- 
egies under uncertainty come together. The cases we cover are 
as follows: 


1. How to make education and career choices when the jobs 
available in 20 to 30 years’ time are uncertain in nature and 
scope. The impact of automation and advances in human level 
machine intelligence (HLMI) will have an impact on jobs, both 
positively and negatively. Some forecasts will prove to be right 
and others horribly wrong. How do you navigate your way through 
educational investment and career decisions now to avoid a high 
cost of error and have a good chance of realizing your career 
goals? Our team of young Oxford researchers encouraged us to 
tackle this problem. Career choice has elements of level 3 and 
even level 4 uncertainty. 


Professor Sally Cripps, Professor of Statistics, University of Sydney. 
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2. How to plan for far horizon events: Will | outlive my savings? 
This is one of the highest uncertainty events we face as individu- 
als. It's a relatively recent problem as people used to retire at 65 
and have a life expectancy of 72. Now they often retire at 60 and 
live to 85 or more. Probability assessment is a key part of tackling 
this level 2 issue. 


3. How you make really long-term investments with growth 
options. Long time frames for investments like infrastructure call 
for particular attention to uncertainty and the role of develop- 
ment options. We take a resource company investment example 
at a level 3 uncertainty to illustrate how our tools are put to work. 
This is one where Rob can look back and see how it played out 
over more than two decades. 


4. How you build new businesses with a staircase strategy 
approach. Where you set out to build new businesses in rap- 
idly changing environments, there is a need for a step-by-step 
approach that simultaneously buys information to reduce uncer- 
tainty and builds new capabilities to reduce risk. Charles and Rob 
developed this approach with colleagues at McKinsey.’ It has 
proved valuable to large businesses and start-ups. It features 
level 2 uncertainty. 


5. How a complex portfolio of strategies in a highly uncertain 
environment are constructed. High uncertainty settings often 
require selecting multiple paths to a long-term goal and making 
trade-offs among strategies as the real world unfolds. Charles 
used these techniques at the Moore Foundation when manag- 
ing a large portfolio of grants that employed often-competing 
strategies to preserve Pacific salmon. The issues here were 
mostly level 3 and 4 uncertainty. 


Let's look at each of these case studies. 


*Mehrdad Baghai, Stephen C. Coley, and David White with Charles Conn and 
Robert McLean, “Staircases to Growth,” McKinsey Quarterly 4 (November 1996). 
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Case Study: How Should | Choose My Career? 


As automation’s and artificial intelligence’s roles in the workplace 
expand at ever increasing rates, predictions for the labor mar- 
ket have many facets of uncertainty compared to 25 or 50 years 
ago. The tailwinds are around mostly non-routine, cognitive 
jobs that didn’t exist 10 years ago—you may have seen the World 
Economic Forum new jobs list: app developer, social media man- 
ager, driverless car engineer, cloud-computing specialist, big-data 
analyst, sustainability manager, YouTube content creator, drone 
operator, millennial generation expert. In 10 years the list will be 
different again. 


So as a young person choosing a career, how would you best posi- 
tion yourself in this changing labor landscape? First, we build a tree 
to capture the economic labor landscape (Exhibit 8.3). It shows 
clearly that the employment share of non-routine cognitive jobs is 
growing, as are non-routine manual jobs, while routine cognitive 
and routine manual job shares are shrinking in share of employment. 


You can use your inside knowledge about your personal level of 
ability, interests, and risk tolerance to guide your education and 
career choice decisions. Jeff Wald, the cofounder and president of 
WorkMarket asks, “What are you passionate about? Does that map 
to what skill sets are needed?”"' To do this, we begin with a simple 
chart to fill in. Each horizontal row in the chart represents a broad 
potential field or sector in which you may end up working; there are 
four vertical columns: The first is the current economic predictions 
for the field, the next are your personal assessments of your ability, 
interest, and ability to take risks. The first column can be completed 
and updated using economic predictions. 


McKinsey Executive Briefing. Technology, Jobs, and the Future of Work. www 
.mckinsey.com/global-themes/employment-and-growth/technology-jobs-and- 
the-future-of-work. 

""The Digital Future of Work,” McKinsey Global Institute, July 2017, https://www 
.mckinsey.com/featured-insights/future-of-work/the-digital-future-of-work-what- 
skills-will-be-needed. 
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DATA ABOUT CURRENT LABOR MARKET 


T A inthe 
Skills Existing Job employment Pay 
Titles share 
Problem J 
solving 
Creativity and Management 
design 
Large, 
Cognitive n” Professional growing High 
inking 
Abstract Research + 92% 
Non-routine thinking 
Collaboration 
H 
PI 
OCCUPATIONAL M | Pa Service Medium, 
GROUP TYPES anes Caring : growing Low-medium 
onstruction 
Execution + 16.1% 
| | | 
Routine 
Analysis Back Office Medium, 
Cognitive cinta ; Administration shrinking Low-medium 
pplication o 
knowledge |/Some professions = TI% 
— H q jm 
Dependability 
Medium, 
Manual [Communication Manufacturing shrinking Low 
Craftsmanship - N% 
p 
EXHIBIT 8.3 


Source: St. Louis Fed, Jobs Involving Routine Tasks Aren't Growing (2016); 
Jaimovich & Siu, The Trend Is the Cycle: Job Polarization + Jobless Recoveries. 


For each potential field, subject area, or employment sector, you fill 
in your self-assessed strength, interest, and risk-tolerance levels in 
the second, third, and fourth columns. You can choose low, medium, 
or high, to rank your preferences. Exhibit 8.4 shows a filled-in matrix 
for one of our research team members. 
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CAREER CHOICE 


Economic projection for 
Sector/Skill area this sector My strength level, My interest , My ability to 
(time horizon and error) (L. M. H) level take risks 
(L. M, H) 
Art/Design M H H 
Business/Finance M 
Health/Human 
services H 
Math/Science H 
Social Science H M 
Peoplel 
Interpersonal M M M 
Trades/ Technical 


EXHIBIT 8.4 


Now how does this information help lead you to a decision? 
You can start by eliminating or pruning sectors in which you have 
low strength or abilities and low interest. Beginning with the most 
promising sector, the sectors for which you have the highest inter- 
est and ability, you can use this tree to guide your next step or 
action. We define three potential strategies or actions: 


e Make a big bet and embark on a path with considerable risk. 

e Make a no regrets move and gain a base level of education or 
training in a safe field that safeguards you from risk. 

e Hedge your bet by investing time and energy in education or 
training for two or more fields or positions. 
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NOW, TAKE ACTION... 


Beginning with highest-ranked field with respect to individual strengths and interests 


Are my strengths, risk tolerance, and interest 
aligned with economic opportunity? 


Yes No 


Entrepreneurial venture Can | make a no regrets move 
& pursue a basic level of 
education in this field? 


No Yes 
(due to my situation) (list skills received) 
Move to next highest To gain employment, does 
ranked field this field require further or 
additional skills? 
No Yes 


(justify with data) (list. skills required) 


Can | acquire the 
skills? 
No 


(due to my situation) 


Yes 
(list skills received) 


Move to a related 


Is hedging field 
necessary? 
No Yes 
(based on field uncertainty) 
Is that a big bet? Pursue additional skills 
No ven in next highest ranked 
(based on field projections) field 
No action required immediately; Make a pivot plan across a 
monitor the field, your specific time horizon to the next 
interests, and economic highest ranked field 
projections over the medium to 
long run 


EXHIBIT 8.5 
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Within these strategies you can see two broad paths: one entre- 
preneurial and the other educational. When faced with high ability, 
high interest, high risk tolerance, and high economic opportunity, 
entrepreneurship may be calling out your name. However, it is often 
more likely that all your stars may not be in perfect alignment; in 
this case, you may need to make a no regrets move and obtain a 
base of education, or choose to hedge your bets across two fields 
with strategies such as double majoring or continued education 
while working. 


Exhibit 8.5 illustrates these strategic choices. 


The tree, table, and series of action trees shown in Exhibit 8.5 can 
easily be applied and iterated upon by an individual while they 
are in the process of making their education or career choice, 
making decisions appropriate to their passions, skills, and risk 
tolerance. 


Case Study: Will My Retirement Savings Last? 


At the other end of life planning, we also find uncertainty. In the 
past, retirement savings typically only needed to last 10-15 years. 
Now the need for retirement income is often stretching out to 
25-35 years, where early retirement is taken and longevity is 
improving. Many retirees now face the question “Will my retire- 
ment savings last?” The answer to this question depends on the 
longevity runway you're on and the income pool you can gen- 
erate, as well as your risk preferences (see Exhibit 8.6). As life 
expectancy increases, and returns on conventional savings remain 
modest, an increasing number of people risk outliving their sav- 
ings. On the minds of people approaching retirement are these 
questions: 


e How should | plan for this uncertainty? 
e Are there actions to take now to mitigate this risk? 
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<— Actual end of runway 
could be closer to 89-91 


<— Predicted end 
of runway 


What we plan for 


EXHIBIT 8.6 


e Are there decisions that can be deferred to a later point in time? 


e Do | need to go beyond my financial planner or accountant 
for advice? 


We admit this isn’t the kind of issue you want to deal with every day, 
but there is a way to do it simply and comprehensively. It requires 
you to estimate your longevity runway, adjust for bias, and take a 
view on your risk preferences in investment strategy. 


We start with a first-cut calculation to solve this problem. In order 
to work out whether you face a risk, you can simply calculate how 
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far your retirement nest egg will go, measured in years of target 
income, and compare that to how long you can expect to live 
given your age. This can be as simple as dividing your retire- 
ment savings by your target retirement income less expenses, 
adjusting for any defined benefit pensions. That is your years of 
retirement runway. 


This heuristic is a good starting point to understanding your situa- 
tion but doesn’t provide the whole answer. Why? By definition, not 
everyone is going to reach average life expectancy and individuals 
have different risk tolerance. Also, your nest egg can be invested in 
different assets that earn more or less, and that have more or less 
volatility. The analysis needs to be taken further than the heuristic in 
order to account for these factors. 


Sketch Out Your Longevity Runway 


Let's look at your longevity runway more closely. If you are a healthy 
35-year-old, your life expectancy is 80 years, but if you make it to 60, 
the average life expectancy increases to 82.8 years." 


At age 60, there is a 50% chance of: 


e A male exceeding 85 
e A female exceeding 88 
e Atleast one ofa male and female couple together exceeding 91" 


12Personal assets and savings are an important part of retirement income in devel- 
oped countries. Government practices vary widely in allowing personal assets to 
be held in addition to a pension. Some countries require assets to be included in 
pension calculations; others, like Australia, permit them to be retained outside 
pension calculations. As a result, your calculations will be different depending on 
the rules that apply where you live. 

Swww.helpage.org/global-agewatch 

“Australian Bureau of Statistics. 
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Planning is typically done for couples, and what is relevant is the 
age at death of the surviving partner. It is another case where the 
joint probability is the relevant calculation, and you can find tables 
online for this joint-longevity question. In this case, the likelihood 
of one or the other living to over 90 is 50%, a 30-year planning run- 
way, almost as long as a working life that starts at 22 and finishes at 
60! The irony is that people tend to underestimate their life expec- 
tancy by upward of 4 years." If you have grandparents who are in 
their 90s, your life expectancy increases. If you have a healthy life- 
style, it is higher again. If you live in a town with high environmen- 
tal quality, it reduces respiratory and cardiovascular illnesses and 
other diseases. If your runway has all the positives, you need to be 
planning to live longer. 


Consider Your Risk Tolerance 


Volatility around investment returns for your retirement account 
movements creates income uncertainty. Different individuals will 
have different risk tolerance for how to handle this uncertainty. 
If you believe you will have high longevity and you have low risk tol- 
erance for depleting your savings, then you could consider buying 
an annuity that covers your lifetime and making adjustments to 
your budget to balance spending to income. Buying an annuity is 
a classic insurance policy to handle uncertainty. You get peace of 
mind around having savings that last a lifetime with an annuity, but 
trade that certainty off with lower returns. On the other hand, if you 
anticipate living a long life and have a tolerance for risk, you should 
consider holding a growth portfolio for investing, rather than what 
is termed a balanced fund. 


Decide What Actions to Take 


The possible actions you might take are shown in the decision tree 
in Exhibit 8.7. 


International Longevity Council UK, 2015. 
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NOW, DECIDE WHAT ACTION TO TAKE... 


eed Ses Possible Actions 
runway? tolerance? 
High Growth fund 
Long 
TES Low —— Annuity 
High Balanced fund 
Yes Medium = i 
WILL | vA Low ——— Balanced fund + Annuity 
OUTLIVE MY N 
ao 
SAVINGS2 ae Short one 
No None 
EXHIBIT 8.7 


There is a surprising conclusion here that we need to explain. It 
has to do with longevity and compounding. Most financial plan- 
ners we know recommend balanced funds for clients approach- 
ing retirement that provide a measure of safety against stock 
market volatility to provide stable current income, typically for 
investors with a 10- to 20-year investment horizon. But what if 
that investment horizon at age 60 is 25 years or even 35 years? 
The simple table below helps to show why the duration period 
of an investment and the power of compounding returns are 
so important. 


The returns from a growth fund are 34% higher for the 25-year 
period and a staggering 51% higher if the investment period is 
35 years (Exhibit 8.8). But you have to have some tolerance for 
greater annual variability in income after retirement. The point is we 
have uncovered a different way to think about the problem of the 
adequacy of retirement savings, one that may feel counterintuitive. 
Moreover, starting down this problem solving path allows a retiree 
at 60 to switch funds if their circumstances change, such as having a 
need for income earlier. On the other hand, you're unlikely to make 


Case Study: How to Make Really Long-Term Investments 


INVESTMENT OF $100K AT AGE GO 


Life expectancy 85 99 


Balanced Fund (40% stocks, 60% equities) 


Weighted return 7.73% ae saam 


Growth Fund (60% stocks, 40% equities) 


Weighted return 9.0% Poeem seam 


EXHIBIT 8.8 


Source: Thomas Kenny, thebalance.com, June 23, 2017, for long-term returns 
(1928-2013) on stocks of 11.5% p.a. and T bonds 5.21% p.a. 


a switch from a balanced fund at 80+, when the longevity runway 
is shortening. 


We have reached a conclusion here that you won't hear from many 
financial planners, to consider investing for growth as you enter 
retirement if you expect a long life for yourself or your partner. 
When we add risk tolerance, a different strategy emerges than 
hedging by buying annuities. Heuristics like a longevity runway and 
compound growth get us to a rich solution set to an all too real 
problem. There is also the need for many to increase their savings 
rate to extend the runway. That should be addressed along with 
asset allocation to avoid outliving your savings. 


Case Study: How to Make Really 
Long-Term Investments 


Investments like bridges, mines, roads, and infrastructure have 
long lives. There is uncertainty over the future operating environ- 
ment that has to be factored in to the problem solving up front. 
There is the need to deal with a range of possible outcomes, which 
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often requires flexibility to address the value of future development 
options that are either enabled or blocked by the decision today 
(another kind of conditional probability). We have chosen to illus- 
trate how to address these issues with a real example of how the 
giant Australian company BHP approached making a long-term 
investment in a mining venture in the past. 


Resource companies seek to find or acquire reserves that are 
low on the cost curve (a tool we introduced in Chapter 3 with the 
climate change abatement cost curve), high quality, and have long 
lives. This is easy to say but hard to do in practice. Low-cost sup- 
pliers like BHP expect to be able to sell their output in good times 
and bad. And they usually have to plan on real prices declining, 
as they have for the past century. Rob led a team advising BHP 
on the acquisition of a major mining asset that was low cost, high 
grade, and that had more than 50 years of life. The problem solv- 
ing approach had two distinctive features to deal with long time 
frames and the associated uncertainty. One was to define the key 
uncertainties related to drivers of value of the mine, such as price 
and volume. The other involved the use of scenarios to model the 
range of possibilities. 


Value Including Development Options 


The team developed a typical valuation tree for BHP by estimating 
net cash flows from the existing mine and discounting them at the 
cost of capital."® You will note that this tree is similar to the return 
on investment trees we developed for Truckgear and Hechinger 
in earlier chapters. The math is straightforward and you can find 
online net present value (NPV) spreadsheets in Excel and other 
software tools. The cost of capital was based on the risk profile of 
the project, the cost of raising capital for BHP, and assumptions 
made about terminal value for a declining resource. 


Tim Koller, Marc Goedhart, and David Wessels, Valuation: Measuring and 
Managing the Value of Companies, 5th ed. (Wiley, 2010). 
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BHP VALUE 


x 
Price 


Profit naran ~ 
s tonne 


Operating cost/ 


Profit tonne of ore 


A margin 

Newent Tx Tons mined 
flows from xa x 

existing mine Orelwaste ratio 


Overhead costs 


NET PRESENT VALUE Capital expenditure 


(Cash flows discounted 
at the cost of capital) 


Key uncertainties 


Net cash flows 
from development 
options 


EXHIBIT 8.9 


The tree Rob's team used for this analysis has an additional branch 
that we termed the value of development options (Exhibit 8.9). 
We added it because we cleaved the problem around existing 
assets and future growth or development options. What made the 
development options so important in this instance was that they 
could be brought “into the money” at a relatively low exercise 
price. Why was this the case? It was because they were able to pig- 
gyback the infrastructure of rail and port facilities already in place 
for the existing mine. Whenever you are valuing long-lived assets 
in uncertain situations, think about what future projects are made 
easier or harder by the moves you make today—sometimes these 
can have more value than the immediate decision at hand. 
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Four major uncertainties had to be addressed to estimate the 
net present value in this case: the world price for the mineral, 
the Australian exchange rate versus the US dollar, the volume of 
ore production, and the value of development options. The volume 
of ore is in a much smaller range of variation than the other factors, 
so received less attention in the analysis. There is a range of future 
outcomes that make it level 3 uncertainty. 


Scenarios for Major Uncertainties 


The use of scenarios in commodity forecasting and resource 
investment has a long history and predates the study Rob led. 
Nowadays large-scale computing has made it easier to run 
large Monte Carlo simulations, and to more accurately measure 
the value of options using Black-Scholes and other valuation 
approaches. 


Our approach was simpler. We had time pressure from the com- 
petitive auction for the asset, and we were confident we could 
address the uncertainties without the need for a large-scale sim- 
ulation model. We looked at the demand/supply balance on the 
world ore market 10 years out in order to generate a range of price 
forecasts. These were based on how the industry cost curve (supply 
curve) would look in the future based on current and expected cost 
positions, reserves coming on stream, and opportunities to reduce 
costs. A range of demand estimates for the commodity 10 years 
forward was overlaid against the cost curve to generate a likely 
pricing band. We also considered extreme cases, such as a price 
level two standard deviations lower and two standard deviations 
higher, cases that have 95% confidence intervals.” We overlaid a 
range of foreign exchange rate assumptions, as financial hedging 
wasn't possible 10 years out. We were then able to put together a 
matrix of net present value calculations for the price, volume, and 
exchange rate assumptions (see Exhibit 8.10). 


"Confidence intervals for events we don’t know are often way too low, a point 
made in M. H. Bazerman and D. A. Moore, Judgment in Managerial Decision 
Making (Wiley, 1986); This can be a big issue for level 3 and 4 uncertainty settings. 
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BHP MINE: VALUATION SCENARIOS 


World cost curve 


10 years out Payoff table 


| 
Price i Ten-year 
jdemand band 


Ore 


+20 Bonanza 
Price 


Average Tee ee AA 
cA Most “Likely” 
Pe ea ee A Range 


| Current 


| | | | | 20 Disaster 


09 08 07 06 
ne Exchange Rate 


I 


Economic scenarios 


Scenario Real Exchange Rate Assumption 

1 AUD declines to 0.9 Unanticipated structural improvement 

2 AUD declines to 0.8 Inferred REER for purchasing power parity 
3 AUD declines to 0.7 AUD correctly valued 

4 AUD declines to 06 Unanticipated structural decline 


EXHIBIT 8.10 


Displaying the results of our analysis in the payoff table allowed 
discussion with the client of how they felt about the assumptions, 
particularly where we ended up in the best-guess range. Impor- 
tantly, it put on the table for consideration the possibility that there 
was a truly worst-case disaster, where price fell precipitously and 
the real exchange rate appreciated strongly. This combination was 
considered highly unlikely but not impossible. The disaster case is 
another joint probability instance—what is the likelihood of prices 
plummeting and the exchange rate being extremely unfavorable. 
If each has a 10% chance of occurring, the joint probability of both 
events is only 1%. The same is true of what we called the bonanza 
case. This is a case where the price is two standard deviations above 
the mean and where the exchange rate is highly competitive. 
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You might say why only look at 10 years out with a 50+ year life 
asset? We felt it was a conservative approach to valuation in an 
uncertain and cash-constrained environment. We incorporated a 
terminal-value estimate by discounting the cash flows after year 10 
at a much higher cost of capital (20%), compared to 10% for the first 
10 years. We ended up being comfortable in recommending the 
acquisition, as the NPV of the initial asset purchase was higher than 
the purchase price under all likely scenarios. 


We have discussed biases to good problem solving at length. 
Large-scale investments, such as company or major asset acqui- 
sition, are ones where biases can creep in, particularly confir- 
mation bias. People get excited in auction situations and want 
to win the bidding contest. Besides the analytic efforts to use 
scenarios, good problem structuring, and sensitivity analysis, we 
had a team environment where the client was genuinely seeking 
a dispassionate external view as input to a board of directors’ 
decision. On the team, we had analytical strength with propo- 
nents, and a divergent view from one team member who took 
the antithesis perspective. These were our efforts to address 
confirmation bias and realistic worst cases, similar to what we 
suggest in Chapter 4. 


Look backs are a tool corporations use to assess how robust their 
capital-investment processes are. Our team did an informal look 
back on this proposal 20 years later from public sources and con- 
cluded that Rob’s original team had been very conservative. We 
didn't anticipate the China resource boom that came 20 years 
later. The resource boom changed both the price and volume 
received for the commodity dramatically. This made the value of 
the development options attached to the acquisition of enormous 
value, such that something close to a bonanza scenario was the 
actual outcome. 


In this case, the value of further development options is an impor- 
tant way to ameliorate the risks of uncertainty. In BHP’s case, the 
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development options were estimated to be 22% of the NPV of the 
acquisition, an important component but one that our look back 
showed was far too conservative (Exhibit 8.11). In hindsight, we esti- 


mate the value created was in the order of billions of dollars above 
the value of the core mining asset BHP acquired. 
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EXHIBIT 8.11 
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Case Study: Building New Businesses 
with Strategic Staircases 


There are many ways to develop company strategies, depending on 
the industry context and company characteristics. Michael Porter's 
5-Forces map or the McKinsey Structure-Conduct-Performance 
model looks at industry dynamics to evaluate entry.'® Dixit and 
Nalebuff focus on mapping competitive game evolution, !? like our 
CSIRO example from Chapter 6. The framework we introduce here, 
which we call growth staircases (or horizons, in some publications), 
is directed at determining the steps that allow successful companies 
to edge out into new businesses in uncertain environments, and 
emphasizes learning, buying options, and building capabilities. 


The origins of the staircases and horizons framework were when 
Charles, Rob, and McKinsey colleagues were trying to come to 
grips with a new business growth aspiration for an energy client.” 
We marshaled a number of cases of how companies had built valu- 
able new businesses in a little over a decade. We started drawing 
out the initiatives they took to enter markets, hire skilled people 
in unfamiliar areas, and make acquisitions, as steps on a staircase. 
We analyzed what steps were taken in what order to build capabili- 
ties, add assets, and reduce uncertainty. The staircase approach we 
visually set out involves: 


e Working backward from a planned outcome, thinking through 
the capabilities, assets, and business learning required to be 
successful. 

e Taking a step-by-step approach to strategic moves, allied to 
the uncertainty level, expanding commitment as uncertainty 
is reduced, 

e Framing in time the staircase steps, knowledge captured, and 
capabilities that are being built. 


18M. E. Porter, Competitive Advantage: Creating and Sustaining Superior Perfor- 
mance (New York: Free Press, 1998). 

"Avinash Dixit and Barry Nalebuff, Thinking Strategically (Norton, 1991). 

?0Our Australian office colleagues David White and Mehrdad Baghai, co-authors 
of The Alchemy of Growth (Perseus, 2000), were important partners in this work. 
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To illustrate, one of our favorite examples of staircase strategies 
is the way J&J built a global contact lens business in a little over a 
decade. This was done in a series of initially small steps, then larger 
and more expansive moves. The staircase and associated capability 
platform are shown in Exhibit 8.12. 


JOHNSON & JOHNSON CONTACT LENSES 


STAIRCASE 
1981 1982 1983 1984-88 
Contracted with 
NYPRO, leader 
in precision 
Acquired rights injection 
to new moulding moulding, to 
process improve costs 
Acquired Cut 2-step and technology 
Frontier process to 1 10-fold 
Contact Lenses step with UV- 
(renamed cured polymer. 


Vistakon) with 
5% of contact 
lens market 


Platform of Capabilities and Assets 


Contact lens 
manufacturing 


EXHIBIT 8.12 


Rights to Relationship 

new lens with leading 

technology injection 
moulder 


1988-92 

Disposable 

lenses 

launched in 45 
Invested $250 countries 
million in Introduced 
development Surevue 
spending over 5 2-week daily 


years with top 
management 
support 

Major launch of 
J&J disposable 


contact lenses 


Advanced 
manufacturing 
process 


wear lenses in 


1991 


Worldwide 
marketing and 
distribution 
network 


1993-95 


Introduced 
Acuvue daily 
disposable 
lenses in 1994 
after another 
$200 million 
investment 
Achieved 
$600 million 
in worldwide 
revenues in 1995 


Soft-molding 
manufacturing 
process 


Source: Mehrdad Baghai, Stephen Coley, and David White, The Alchemy of 
Growth (Perseus, 1999). 


It's one thing to see a staircase in hindsight like J&J's, but how do you 
go about building one from scratch? How do you deal with uncer- 
tainty about the success of staircase steps and competitor action? 
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We outline the approach with a discussion of staircase architecture 
and an example of a staircase under construction. 


Staircase Architecture 


There are three considerations in staircase architecture: 


1. Stretch: The distance of the step-out from the established 
platform, the complexity of new capabilities targeted by a step- 
out including the degree of integration required. Whether the 
stretch is too great is a balance between the ability to absorb 
new skills and the competitive demand for speed to establish 
market presence. 


2. Momentum: The positive effect of early success, often based 
on small moves, on the learning and confidence of the organiza- 
tion. Momentum becomes a critical factor in markets that feature 
increasing returns to scale, involve de facto standard setting, or 
where winners take all. 


3. Flexibility: Maintaining fluidity in the face of uncertainty has 
several features, including picking step-outs that generate the 
greatest option value for follow-on initiatives and being pre- 
pared to run more than one horse in the race, as we discussed 
in the Pacific salmon strategy example. Part of maintaining flex- 
ibility is avoiding creating sunk assets that could later become 
stranded. This can be achieved by variable cost structures like 
contracting and outsourcing. 


Staircase architecture involves bringing these three considerations 
together to work through choices about staircase steps and how 
they are sequenced (Exhibit 8.13). Teams have to debate the trade- 
offs between smaller versus larger steps, option-laden moves 
versus focused commitments (that may be better at blocking com- 
petitors), and the speed of moves that can create momentum— 
but may be challenging to integrate with the existing business. 
The usual techniques of role-playing and competitor assignment 
enrich the team discussion of the planning of staircase steps to 
be taken. Google has investments in hundreds of new ventures. 
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That’s an approach where they place many small and medium bets 
in interesting spaces, and often multiple horses in the same race. 
By contrast, a start-up may have one route to market with no more 
than a couple of options, a focused commitment necessitated by 
available venture funding. 


STAIRCASE ARCHITECTURE: 3 KEY DRIVERS 


GROWTH 
VISION 
ACHIEVEMENT 


Degree of posses 
Momentum H 


Ensuring 
Flexibility 


PLATFORM DEVELOPMENT (necessary competences, assets, relationships) 


Extent of 
Stretch 


EXHIBIT 8.13 


Staircase Under Construction: Ripper Group Example 


Staircases can be used by start-ups to map the business goals and 
to show the desired strategic path to get there with learning steps 
and capability building. The value of working backward then for- 
ward is to gain agreement on what capabilities need to be built, 
and in what order. Consistent with having one-day answers and iter- 
ating in the seven steps the aim is to agree in detail on the staircase 
steps for the next 12-18 months, take stock, and then plan the next 
12-18 months as new information on the market and the capability 
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trajectory come to hand. The end goal or its timing is then altered 
to reflect the market and competitive realities. Agile development, 
failing fast, and sprints are the hallmarks of this approach to strate- 
gic problem solving. 


The Ripper Group shark-spotting algorithm was discussed in 
Chapter 6. Ripper is an Australia-based company started by Kevin 
Weldon, who was president of International Surf Life Saving, 
and Paul Scully-Power, the first Australian to journey into space. 
Ripper’s goal is to be a global leader in emergency services 
employing unmanned aerial vehicles (UAVs) or drones, building 
from their current competence in beach life saving toward flood 
rescue and damage assessment, and bushfire fire detection and 
response. National security events and other natural disasters fall 
within their scope. 


Their initial staircase steps for Ripper are similar to those faced by 
most new ventures—start with a paying customer and seek to boot- 
strap its way to building operational capability (see Exhibit 8.14). 
Make some early judgments about the ecosystem you are in 
and how you plan to compete. In this case they chose to have a 
customer focus around emergency services, based on having dis- 
tinctive operational capability, and to have a software rather than 
hardware focus. 


The Ripper Group views itself as a player in an ecosystem of early- 
stage customers, regulators, hardware suppliers, and software and 
data analytics. This has taken them down a path of collaboration 
with the principal regulator to address issues of safety and per- 
formance, such as operating beyond visual line of sight (BVLOS) 
and night flights. In order to fulfill their mission of saving lives on 
beaches, they collaborated with machine-learning scientists to 
develop the world’s first shark-spotter algorithm, software that dis- 
tinguishes drone data streaming between a person, a dolphin, and 
a shark, a visual of which we showed in Chapter 6. 


Their staircase shows the foundations and capabilities they have 
built in a couple of years and the major moves they plan to reach 


Case Study: Building New Businesses with Strategic Staircases 
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the goal set by 2020. In an arena where entry barriers are currently 
low, they are focused on building a reputation as trusted opera- 
tors who are able to take on the most difficult assignments. This 
requires that they continue to build capabilities related to flight 
operations, payloads, and data analytics. They are responding to 
opportunities in related vertical markets such as the risks mines face 
with tailings dams, asset inspections for utilities where flood or fire 
damage has occurred, and agriculture use of drones for weed con- 
trol. Uncertainty is level 2 or 3, as regulation on drone use could 
encourage or curtail market potential. 


Further out on the Ripper Group staircase are measured steps, 
incomplete at this point as you might expect, to achieve global 
sales and distribution. Other steps are to deepen cloud and analyt- 
ics skills so they have distinctive technology capabilities. The Ripper 
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Group is invited regularly to go to other countries and replicate 
what they are doing in Australia. At this point they have neither 
the resources nor the time to do that without local partners. One 
idea is to follow the lead of other software suppliers and provide a 
package of software and operational capability that they call drone 
in a box for emergency services. International customers have the 
opportunity to visit their training academy to test out payloads 
for different applications and be trained in innovations such as 
BVLOS, LIDAR, non-GPS flights, and the use of ML algorithms in 
emergency services. 


Staircases allow visualization of a growth plan, with active debate 
about the size and scope of staircase steps. They are planned in line 
with reducing uncertainty over time of the market evolution and 
regulatory environment so that the enterprise doesn't get ahead of 
itself. Flexibility is built-in: As new information comes to light, for 
example, on regulation like BVLOS or night flights, the staircase is 
redrawn. Similarly, early success with international sales may lead to 
accelerating global rollout of products like the beach of the future 
we described in Chapter 6. 


Case Study: Managing a Long-Term Strategy 
Portfolio, Pacific Salmon Example 


The longest term and most uncertain problems to solve are those 
in the social and environmental space. Our Pacific salmon case, 
which Charles worked on for a dozen years, is a great example. The 
scale of the problem was literally oceanic (covering four countries 
and the entire North Pacific ocean) and multidecadal, and affected 
whole ecosystems—for which Pacific salmon are an apex or foun- 
dational species—and huge numbers of people, both in livelihoods 
and culture. This is a big one! 


The tools we introduced in earlier chapters are entirely applicable 
here, and the team made frequent iterative use of them over the 
fifteen-year project. But some other tools, particularly related to 
managing portfolios of strategies, were also important to cracking 
a problem of this scale. Let's take a look at them. 
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Theory of Change Map 


For large, multiyear problems it is important to have an overall 
theory of change (TOC)—the strategic map for visualizing social or 
environmental change at this scale. In the case of the Pacific salmon 
team Charles worked with, that TOC map was an evolved variant of 
the logic tree introduced in Chapter 3. 
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As you can see in Exhibit 8.15, this map summarized the team’s 
overall strategy for preserving salmon and salmon ecosystem 
function at the North Pacific scale. This included strategies for 
(1) preserving habitat integrity, (2) neutralizing the threat from 
open-net pen aquaculture, (3) mitigating the impact of hatch- 
ery propagation, and (4) ensuring sustainable fisheries manage- 
ment. Those high-level strategies led to more than 300 grants to 
nearly 100 organizations over a decade and a half—a lot of detail 
to manage. 


Portfolio of Strategies Map 


One of the ways the team managed and communicated about the 
large portfolio of strategies and investments was to visualize them 
in a matrix that captures both the stage of investment and the level 
of aspiration, expressed as incremental or transformational change 
(see Exhibit 8.16). 


The team wanted to balance the investments it was making in 
Pacific salmon ecosystems across the logical stages of change: 


1. Seed: Those strategies designed to create early condi- 
tions for change, which included some fundamental science 
investments to build understanding among government 
management agencies, early-stage habitat protection invest- 
ments based on land planning and First Nations’ ownership 
rights, and mobilizing a base of support for other long-term 
change efforts. 


2. Cultivation: Those strategies where interest groups had come 
to the table and were willing to work together on solutions for 
both ecosystems and economic users. These are mostly invest- 
ments in multi-stakeholder negotiations to change older status 
quo resource harvest regimes. 


3. Harvest: Those investments to cement conservation gains and 
solidify institutional support for new conservation centered 
resource management processes. 
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And the team wanted balance between those initiatives that had 
transformational aspirations—expensive and risky—and those that 
aimed for incremental but important change. 


This visualization of a complex portfolio made it easy to see if the 
balance was off relative to the Foundation’s values, which would 
have been true for a set of only high-investment, high-risk strate- 
gies, on the one hand, or a set of sensible workaday approaches 
that would only lead to incremental change, on the other. It was 
an effective picture for trading off the risks of individual strategies 
against others, and for communicating the overall risk and return 
profile to the Foundation’s trustees and senior decision makers. 


229 


230 


Problem Solving with Long Time Frames and High Uncertainty 


Managing Regional and Topical Strategies 


With a problem at this large scale, each of the individual regional 
or topical strategies required its own logical disaggregation, prior- 
itization, and strategy mapping. To give a concrete example, one 
of the regional Pacific salmon strategies that Charles worked on 
was in Northern British Columbia, centered on the massive Skeena 
River watershed. 


The Skeena has a very long history of sustainable aboriginal fish- 
eries before European fishers turned up—and a very contested 
one since then, with different types of fishers fighting for share, 
and also conflict between federal and provincial fisheries man- 
agement agencies. In recent decades, this conflict has been set 
against a backdrop of declining salmon runs and difficult eco- 
nomic conditions for resource users. The federal government 
had developed a science-based management policy for salmon 
fisheries, but failed to implement it. Conflict grew as access to 
salmon and the salmon economy was constrained for fishers, 
and as environmental groups ramped up pressure on managers. 
No one liked the status quo, but finding incremental pathways to 
change was challenging. 


In complex situations of this nature, where many things are broken 
but stakeholders don’t agree on fixes, it can help to set out a vision 
for change that nearly everyone can agree on. This helps warring 
parties to see what element of a vision they have in common, which 
can lead to collaborative problem solving. Exhibit 8.17 shows the 
vision for change in the Skeena watershed that was developed by a 
number of the groups involved. 


While this vision didn't lead to an immediate détente, it did help 
bridge the parties to an agreement to conduct an independent 
science review (ISRP) of conditions in the Skeena watershed. The 
findings of that ISRP helped generate stronger, science-based 
management plans and multi-stakeholder governance conversa- 
tions that led to improvements that are ongoing. 
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For the Foundation team Charles worked with and its allies in the 
Skeena region, it was useful to visualize the map toward change 
as a set of strategies, some of which were competing (that is, one 
or the other was likely to succeed, but not both) and some of 
which were mutually supportive. Exhibit 8.18 is what that picture 
looked like. 
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As you can see, the map is based on a logic-tree disaggregation of 
the big levers for change in the watershed, but it shows the differ- 
ent supporting actions and policies and institutions that were criti- 
cal to drive change. With complex social problems, it is helpful to 
develop detailed strategic maps that help you manage problem 
solving on multiple levels. Game theory trees are also useful here. 


Problems to Try on Your Own 


Uncertainty will remain one of problem solving’s biggest chal- 
lenges. But there is a toolkit for working through your own long 


time frame and complex problems: Frame the uncertainty and 
then iterate through the analysis to decide which actions to take 
to address the uncertainty. Uncertainty is about probability, subjec- 
tive and objective, that brings into play how you pursue options, 
take fair bets, take out insurance policies and hedges and make no 
regrets moves. 


Chapter 8 Takeaways 


Uncertainty is a feature of most difficult problems. 


Understanding the type and level of uncertainty—and your risk 
tolerance—is the first step to problem solving under uncertainty. 


There is a suite of actions to build into your strategy development 
to deal with uncertainty and risk: buying information, acquiring 
low-cost options, paying for insurance, and taking low-regrets 
moves that build capabilities and knowledge. 

The value of long-term options gained or blocked by strategic 
moves in uncertain environments may be a key element of your 
calculations. 

A staircase approach can help companies step-out into new and 
uncertain markets, managing stretch, maintaining flexibility, and 
driving momentum as new capabilities and assets are built. 
Really long term and uncertain strategies require an overall 
theory of change map and a portfolio of strategies matrix to 
visualize and manage families of strategies. 


Problems to Try on Your Own 
1. Place the long-term issues your business or nonprofit faces into 
the five-level schema. 


2. What uncertainty reducing mechanism are you taking for 
each level? 
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3. Take a personal long-term problem and develop the approach 
you might take to resolving it. 


4. For a growth plan you are developing, lay out the staircase for 
the next five years. 


5. For a long-term societal issue you're interested in, try develop- 
ing a theory of change and portfolio of strategies. 


Chapter Nine 


Wicked Problems 


“Wicked problems” is the name given to describe a class of prob- 
lems that seem to defy problem definition and solution because 
they are impossibly linked to complex systems.’ These problems 
typically involve multiple causes, major values disagreements 


‘Horst W. J. Rittel and Melvin M. Webber, “Planning Problems Are Wicked 
Problems,” Polity (1973). 
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among stakeholders, unintended consequences, or substantial 
behavior change in order for the problem to be solved. Terrorism, 
environmental degradation, and poverty are often proffered as 
examples of wicked problems. Wicked problems are different, and 
we won't pretend they are easy to crack. But we believe the seven- 
steps approach sheds light on even these most difficult conun- 
drums. As citizens, we have to have tools to help us understand and 
evaluate the solutions proposed by politicians and policy makers. 


In this chapter we explore the range of problems that have been 
termed wicked problems. These are typically at a societal or even 
global scale. In particular, we take on the “wickedness” involved in 
two case examples, obesity and overfishing of the commons. 


Obesity as a Wicked Problem 


Obesity is a global phenomenon with huge economic, social, and 
personal costs. The McKinsey Global Institute (MGI) estimated obe- 
sity to have a similar social burden ($2 trillion) as smoking, armed 
violence, and terrorism. As the authors put it, “Obesity is a com- 
plex, systemic issue with no single or simple solution. The global 
discord surrounding how to move forward on this growing problem 
underscores the need for integrated assessments of potential solu- 
tions.”* Obesity has dozens of contributing causes and has all the 
elements of a wicked problem with its complexity and interdepend- 
encies.? There are genetic, environmental, behavioral, cultural, soci- 
etal, and income and education dimensions involved. How then do 
you look to solve such a problem? 


The MGI report allows us to see how the McKinsey team approached 
the obesity problem in a way similar to the way we have outlined 
through the book, by defining the problem scope and constraints 
on solutions, disaggregating the problem along familiar demand/ 


Richard Dobbs et al., Overcoming Obesity: An Initial Economic Analysis, 
McKinsey Global Institute (November 2014). 

3Bryony Butland et al., Foresight:Tackling Obesities: Future Choices — Project 
Report, 2nd ed. (UK Government Office for Science, 2007). 
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supply lines, undertaking comprehensive analysis of intervention 
options using a well-known public health framework, and conclud- 
ing with a synthesis and call to action. We explore these elements 
of problem solving in more detail to illustrate how you can make 
progress tackling wicked problems with the seven steps. 


Problem Definition 


The MGI report sets out the dimensions of the problem as follows: 


e Overweight and obese people are estimated to comprise 30% of 
the UK adult population, and the percentage continues to rise. 


e The economic burden in the UK is estimated at $73 billion per 
annum in 2012, behind smoking at $90 billion. 


e The direct medical costs of conditions related to being over- 
weight or obese costs the UK Government $6 billion per annum. 


e Healthcare costs rise with BMI levels (body mass index): Obese 
classes incur 58-86% higher medical costs than individuals with 
normal weight levels. 


The MGI problem definition posited the UK government as the 
decision maker. Government agencies, clinicians, and research- 
ers were consulted in the process. Few constraints were placed on 
solving the problem. Regulatory interventions, such as taxes on 
high-sugar drinks and public health programs, were considered. 
The main requirements were for them to be judged cost effective, 
to have impact, and to have an evidence basis to warrant the inter- 
vention. Healthcare payers were excluded in this report because 
of the NHS role in the UK as a universal care provider. The MGI 
study in 2014 proposed a goal of a 20% share of overweight and 
obese individuals to be brought to normal weight category within 
five years, a demanding target. 


Cleaving the Problem 


Chapter 3 gave examples of a number of ways to cleave soci- 
etal problems. We can cleave the problem around incidence and 
severity, behavioral and clinical perspectives, even financial and 
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non-financial considerations. MGI chose to cleave the problem 
around demand and supply, employing a cost curve, similar to the 
approach used for reducing CO, emissions in our earlier climate 
change example. The value of this cleaving perspective is that it 
provides a menu of opportunities in descending order of cost and 
size of impact. 


Analysis 


The MGI team categorized interventions in 18 groups, such as 
high-calorie food and drink availability, weight management pro- 
grams, portion control, and public health education. In all, they 
looked at 74 interventions. For each intervention, they estimated 
cost and impact in terms of dollars per disability-affected life years 
(DALY), a World Health Organization measure of disease impact. 
They selected 44 interventions deemed highly cost-effective, 
coupled with an evidence rating, that could achieve the 20% tar- 
get reduction in five years. The cost to the UK was estimated at 
$40 billion to implement the interventions. Exhibit 9.1 shows the 
MGI cost curve. 


The cost curve shown helps policy makers in several ways, the most 
important being setting priorities. Typically, you will seek to imple- 
ment the low-cost, high-impact initiatives first, particularly those 
considered small wins, before tackling the more divisive and costly 
ones. On Exhibit 9.1 those are the widest bars toward the left-hand 
side. Over time, the ordering of initiatives will change on the cost 
curve as information is gathered on cost and impact, making it a 
dynamic tool for policy makers. And as we explain below on walk- 
ability, the cost-effectiveness can change for an intervention: For 
example, active transport gets easier to implement if higher prop- 
erty values reduce the net cost of redesign of cities for walking and 
cycling (via property tax funding). 


We would like to have seen some interventions that had nega- 
tive cost, meaning it pays to do them, at the bottom of the cost 
curve, much as we saw with the carbon-abatement curve. The MGI 
analysis points to higher health costs of $750-$1,100 per annum 
for obese people compared to those in the normal weight range. 
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Where these costs are largely borne by a national health system, 
individual incentives are blunted in effecting savings. Nonetheless, 
individuals who can see a way to improve employment opportuni- 
ties and extend longevity may choose to invest in weight loss and 
fitness programs without regulation. We return to the question of 
incentives shortly. 


Synthesis and Call to Action 


The MGI team expressed a guiding purpose to reverse the trend 
of obesity and reduce its personal and societal cost. In an arena 
with multiple players and interests, it was incisive in proposing 
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interventions that were cost effective and, for the most, part had a 
strong evidence base. Having explored a range of choices in terms 
of cost and impact, the report set out a call to action to urge deci- 
sion makers to act now on the portfolio of interventions, rather than 
waiting for an unlikely silver bullet solution. The MGI report empha- 
sized government interventions over other types of action given 
their problem cleaving. Changing human behavior is really tricky, 
but we see a mix of rigor and pragmatism here that is consistent 
with the approach we have set out in the book, one that is useful for 
simple and wicked problems alike. 


Brainstorming Further Ideas on Obesity 


Throughout the book we have emphasized iterations between 
analysis and action. Wicked problems like obesity exemplify the 
need for iteration. When the next iteration of the MGI report comes 
along, we see an opportunity to include additional interventions, 
and particularly behavioral interventions. Here are five propos- 
als that emerged from our brainstorming, analysis, and interviews 
with experts in the field. We have selected them because in prob- 
lem solving we look for holistic approaches, incentives to change 
behavior, and high-leverage initiatives. 


1. Including income and education as policy variables in obesity. 
In the United States, there is evidence reported in a recent Rob- 
ert Wood Johnson Foundation analysis that adult obesity rates are 
“showing signs of leveling off.” This better news is tempered by 
findings that “obesity rates are around 30% higher among adults 
without a college education and with incomes below $15,000 com- 
pared with other adults.”4 These findings are similar to ones that 
came out of the US city-based regression analysis in Chapter 6, 
where income and education, along with walkability and comfort, 
explained 82% of the differences in obesity for the population of 
68 US cities. These findings argue for taking a holistic view and 
linking income, education, and prevention and treatment programs. 


“State of Obesity: Better Policies for a Healthier America. Trust for America’s Health 
and Robert Wood Johnson Foundation. August, 2017. 
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2. Making more use of incentives for individual action. For some 
time, life insurers have added loadings to policy costs related to 
obesity. Why wouldn't a health insurer be prepared to share savings 
of obesity costs with individuals? We are thinking of an incentive 
payment where half the savings are shared with individuals based 
on obesity reductions. The insurer could invite participants, who 
were deemed suitable for significant weight reduction, to enter an 
agreement with each person who brings BMI back to the accept- 
able range in a 12-month period and retains it for 12 months. 
Bundles of incentives could be tested in focus groups before 
launch of an incentive program. Employers could support this idea 
as well: Some Japanese employers provide vouchers and subsidies 
for public transport use, unlike American employers that typically 
provide parking spaces as an employee incentive. Many employers 
subsidize gym membership in both countries. 


3. Could social networks be a key to unwind obesity? A large- 
scale study of obesity in the United States noted the increase in 
the obesity rate from 23 to 31% over a 32-year period. Their con- 
clusion was dramatic: “Obesity may spread in social networks in a 
quantifiable and discernible pattern that depends on the nature of 
social ties.”° Their findings were that if one person became obese 
the likelihood of a spouse becoming obese was 37% higher, 40% 
for siblings, and 57% for friends. They drew on this connectedness 
to argue that “medical and public health interventions might be 
more cost effective than initially supposed, since health improve- 
ments in one might spread to others.” Could this insight lead to 
the great social unwinding project? And who better to involve than 
the UK Behavioral Insights Team, the so-called nudge unit. Where 
would they start? In the same way that one person becoming obese 
impacted others, one person as a spouse, friend, or sibling could 
take the lead and be supported by committing to regaining nor- 
mal weight. We now know a lot more than we did a decade ago 


5Nicholas A. Christakis and James H. Fowler, “The Spread of Obesity in a Large 
Social Network over 32 Years," New England Journal of Medicine 2007, no. 357 
(2007): 370-379. 
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about social networks; tackling obesity with this knowledge seems 
a sensible avenue to explore. 


4. Supporting high-leverage programs such as reducing weight 
gain in pregnancy and early childhood. The ramifications of exces- 
sive weight gain in pregnancy act like a chain reaction over three 
decades or more. For the 40% of pregnant women in the UK who 
enter pregnancy overweight or obese, there is a 33% increased risk 
of obesity in the child at birth and at six years, 40% of overweight 
children continue to be overweight in adolescence, and 75-80% 
of obese adolescents will become obese adults.’ The cumulative 
effects over a lifetime are huge. There is research under way to 
explore whether interventions for expectant mothers can reduce 
weight gain in the gestation period (6-20 weeks). If this can be 
demonstrated in randomized control trials, we have another espe- 
cially high-leverage opportunity, and likely low-cost intervention. 


Professor Louise Baur, a researcher who was a member of the WHO 
Commission on Early Childhood Obesity, cites a review of 17 stud- 
ies where infants who experienced rapid weight gain from zero to 
two years were almost four times more likely to be overweight or 
obese later in life. She recommends specific strategies for provid- 
ing nutrition advice to parents, promoting breastfeeding for the 
first 12 months, and having a monthly visit to a doctor to monitor 
weight gain.’ There may be extraordinary leverage that can come 
from simple, straightforward actions to address the issue of child 
obesity and its knock-on effects. 


5. Walkability or Active Transport. Cities like New York or Tokyo are 
good examples of walkable cities. We decided to look at walkability 


éH. T. Tie, et al., “Risk of Childhood Overweight or Obesity Associated with Exces- 
sive Weight Gain During Pregnancy: A Meta-Analysis,” Archives of Gynecology 
and Obstetrics 289, no. 2 (2014): 247-257. 

7US data source provided by Professor Desiree Silva, the ORIGINS project at Joon- 
dalup Health Campus, Western Australia. 

8Submission 10 to Senate Select Enquiry into the Obesity Epidemic in Australia, 
July 2018. 
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in 68 US cities: it is a statistically significant variable in explain- 
ing obesity differences. The relationship is that a 10% increase in 
walkability reduces obesity by 0.3%. For the UK, this would have 
impact similar to a 10% tax on high-sugar, high-fat products. While 
it is on the margin of highly cost-effective interventions in the MGI 
report, it is possible that if it included higher property tax levies it 
could improve benefit/cost ratios. For example, in the top 30 metro 
areas in the United States, “walkable urban places (walkUPs) rent 
at a 74% higher premium/sq. ft. over drivable suburban areas,”? 
suggesting that future property taxes could help fund the conver- 
sion to walkable cities. This isn’t an impossible goal: The average 
Japanese citizen walks double the distance daily of their American 
counterpart. 


Overweight and obese populations pose huge issues for policy- 
makers around the world. While there may be multiple causes— 
cultural, behavioral, socio-economic, and biological—this doesn't 
imply that the problem should be declared wicked and left to one 
side. We favor the reverse approach: setting ambitious targets and 
employing good problem solving approaches at the societal level 
with multiple intervention paths. Our look at obesity has made us 
more confident that we can find solutions than we were when our 
team first discussed this wicked problem. We don’t claim the prob- 
lem is solved. But this work shows a large number of promising 
avenues to fight obesity that have a strong evidence base, and 
reasonable costs relative to the economic burdens this epidemic 
imposes on society. 
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Environmental degradation is one of the most pervasive wicked 
problems. The famous article by Garrett Hardin, “The Tragedy of 
the Commons,” written in 1968,'° influenced many to reach the 


Walk Economy, The Place Report (2016), 7. 
Garrett Hardin, “The Tragedy of the Commons,” Science (December 13, 1968). 
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view that so-called common-pool resources, such as public land, 
water, or fisheries, required either government intervention or 
private ownership to avoid overuse. Elinor Ostrom, winner of the 
Nobel Prize in economics, showed that there are solutions available 
to the problems of the commons, some of which are long-standing 
arrangements among resource users that have elements of com- 
munity management via norms, and elements that mimic private 
ownership via certain kinds of harvest rights."' Let's look at an exam- 
ple of fisheries reform that employed clever problem solving to 
achieve much better outcomes. 


Problem Definition 


The fishery we want to highlight is the US West Coast Ground- 
fish fishery off the coast of California, a portion of a vast fishery 
extending from the coast to 200 nautical miles between the US- 
Canadian border and the US-Mexico border. The fishery had 
been in decline for some time, from a catch valued at $110 million 
in 1987 to a catch of only $35 million in 2003. In 2001 a Federal 
judge ordered the Pacific Fishery Management Council (PFMC) to 
examine the impact of trawl fishing, in effect sweeping the bottom 
of the ocean, on marine habitats. The review confirmed trawling 
was having a substantial negative impact on habitat and species 
diversity. All of the players could assume that Federal regulation 
was on the way after this report—even though most government 
interventions on gear type and fishing access have had limited 
success in slowing resource decline. The California coastal fishery 
had all the classic elements of a common-resource problem: too 
much access and fishing capacity, no safe zones for fish repro- 
duction, gear types that damaged productive seafloor habitats, 
declining populations of fish, and increasingly difficult economics 
for fishers. 


"Elinor Ostrom, Governing the Commons (Cambridge University Press, 1990). 
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CONVENTIONAL SOLUTIONS TO OVERFISHING 
Problem aim Strategy Side effects 


Shift trawling to new areas 
Trawl closure areas 


REDUCE Opposed by fishers affected 
NEGATIVE 

EFFECTS Reduce trawling ————— Federal buyouts viewed as 
OF BOTTOM errant failing to reduce trawl effort 
TRAWLING 


Higher costs 
Modify gear Eee 


Regulatory burden on 
fishers already struggling 


EXHIBIT 9.2 


Conventional Solutions 


While the problem was identified in the PFMC report, the solu- 
tions to reduce the negative effects of bottom trawling were 
difficult to implement. Three strategies had been adopted else- 
where with little success: trawl closure areas, trawl effort reduc- 
tion via permit buybacks, and proposals to modify fishing gear to 
lower impact on species and by-catch. The typical consequences 
or outcomes of each strategy employed in a one-off way are 
shown in Exhibit 9.2. 


Something had to change. Conventional top-down regulatory 
approaches had poor success and the economics of remaining 
fisheries participants were fragile and getting worse. Enter Chuck 
Cook and the Nature Conservancy (TNC). Chuck Cook, a veteran 
TNC staff member and now Senior Fisheries Advisor, was instru- 
mental in much of the restructuring of the California fishery.’ 


Oral communications, Chuck Cook and Charles Conn, August-October, 2017. 
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He and Charles worked together earlier on the conservation of 
Palmyra Atoll in the mid-Pacific. Chuck has deep experience in 
working with local communities, government, and partners to find 
common ground and find viable solutions to sustainable-catch 
issues. Chuck saw a potential solution path in applying an analogy 
from land-conservation easements and market transactions in the 
marine environment. 


New Approach to Restructuring 


A better solution emerged from TNC working closely with fishers in 
a process that started in 2004.13 When Chuck met with the owners 
of 22 trawl permits, he expressed TNC'’s interest in purchasing trawl 
permits on the condition that they support the establishment of a 
marine protected zone prohibiting bottom trawling of 3.8 million 
acres. The discussions progressed between fishers and TNC, and in 
2006 TNC bought over 50% of the permits for $7 million, making it 
the second largest holder of groundfish harvest rights on the West 
Coast. TNC then proceeded to lease the permits back to fishermen 
with conservation restrictions. 


An additional obstacle had to be overcome to encourage fish- 
ers to use less-damaging gear than trawling in the form of hooks 
and traps. In Phase 2 of this project, led by long time TNC staffer 
Michael Bell, TNC entered into voluntary private agreements with 
fishermen that specified where fishing could take place, strict lim- 
its on catch by species and by-catch, and gear type restrictions by 
area. Cooperative fishing, as opposed to highly competitive fish- 
ing between individuals, became the norm in California's central 
coast, resulting in improving the economic and environmental 
performance of this fishery. In 2011 the National Marine Fisher- 
ies Service introduced a full catch share and individual transfer- 
able quota system. Catch shares allocate a total catch by species 


Mark Tercek and Jonathan Adams, Nature’s Fortune (Island Press, 2015). 
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to each permit-holder, with limits on both the target and by-catch 
species harvested. This was the last critical element of the restruc- 
turing of the coastal groundfish fishery. 


This innovative conservation fishing agreement has been likened to 
an easement on land, conferring limited property rights in the form 
of permits and quotas that can be bought and sold with conditions 
on use. These agreements were struck between the Pacific Fishery 
Management Council, fishermen, and TNC. They are based on prin- 
ciples relating to sustainable catch, the science of fish stocks, and a 
collaborative approach between conservationists, scientists, fisher- 
men, and government. They incorporate many of the ingredients 
cited by Elinor Ostrom in her analysis of successful arrangements 
for managing common-pool resources (CPR). 


Morro Bay Case Study 


In response to these changes, a community quota fund (COF) was 
established in Morro Bay, one of the California regional ports, as 
a local non-profit, with a board comprising the fishing commu- 
nity, the city of Morro Bay, scientists, and economists.'* TNC has 
sold most of its permits and harvest rights to this body and others 
like it along California’s central coast that hold them as commu- 
nity assets, leased out to individual fishers. The structure is shown 
in Exhibit 9.3. 


The catch in Morro Bay, one of the four principal locations along 
California's central coast participating in the Pacific groundfish fish- 
ery, was valued at $10 million in 1995, but fell to $2 million in 2003, 
reflecting the loss of fish stocks. Following the new arrangements 
described above, the value recovered to $4 million in 2010 and 
$8.3 million in 2014. These improvements are partly due to recov- 
ery of fish numbers, but derive also from higher price per pound 


“Morro Bay Commercial Fisheries. 2015 Economic Impact Report Working Water- 
front Edition. 
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COMMUNITY FISHERIES OWNERSHIP 


Board 
Fishing Community, City of Morro Bay, 
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EXHIBIT 9.3 


earned in quota fisheries, where catch volume is spread out over 
time and fishers can access higher value markets. Exhibit 9.4 shows 
a near-death experience for a fishery that a combination of regula- 
tion and a market solution via collective action has, so far, been 
able to address. 


According to the Morro Bay 2015 Commercial Fisheries Economic 
Impact Report, “Morro Bay has successfully transitioned from a 
larger fleet reliant on trawl and large volumes of landings to a smaller 
fleet profile with a wide diversity of species and gear types. This is 
evidenced by the growth in earnings from a 25-year low in 2007. 
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EXHIBIT 9.4 


2015 is the strongest year in the last 20."'° Beyond these economic 
measures is one of species diversity and the re-emergence of spe- 
cies, such as lingcod, factors that augur well to sustain the fishery 
into the future. The commercial fishing industry accounts for 195 
jobs for fishermen, dockworkers, and seafood processing in Morro 
Bay. In addition there is tourism that attracts two million visitors 
each year, many of which come to Morro Bay to see a working port 
and to purchase or be served fresh seafood. 


Overfishing may have been addressed at Morro Bay and other 
ports on the California coast, but it remains a huge issue around 


Morro Bay Commercial Fisheries. 2015 Economic Impact Report Working Water- 
front Edition. 
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the world. The examples of common-pool solutions and collective 
action provide encouragement that the problem doesn’t have to 
be left in the too-hard basket. An innovative debt-for-nature swap 
in the Seychelles in 2016, again led by TNC and its NatureVest 
unit, has seen the creation of a marine-protected area (MPA) of 
98.9 million acres, an area the size of Germany. What we see at work 
in California, the Seychelles, and other collective-action settings is 
problem definition, problem disaggregation, and analysis that lead 
to new solutions that work better than conventional regulatory 
interventions. Externalities, one of the defining characteristics of 
wicked problems, can be internalized and neutralized through the 
innovative mechanisms developed by systematic problem solving 
approaches. 


Conclusion 


We have explored several wicked problems and ways to develop 
innovative solutions. The solutions range from relaxing assump- 
tions for what can be part of the solution, being prepared to revise 
incentives to internalize externalities, to cleaving problems differ- 
ently to reveal new insights into interventions. These problems are 
more difficult, with more causes and difficult social issues to over- 
come. But they are just as amenable to systematic problem solving 
as the simpler problems of Chapter 1. 


Are there other wicked problems where there are limitations to 
our problem solving approach? Some of the extreme dimensions 
of wicked problems present particular challenges, but we don’t 
believe that makes them outside the scope of problem solving. 
Here are some examples: 


e Where the problem changes shape as a result of an interven- 
tion. Social problems, such as addressing welfare payments, are 
often of this nature, where the intervention may lead to depend- 
ency and undermine positive behaviors. Theories of change, such 
as we used in Chapter 8, are needed to map feedback loops that 
may be negative and seek to get incentives right. 
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e Where there is no such thing as a single right answer to the 
problem. For example, determining whether nuclear power has 
a role in a country’s energy mix: We accept that there may not be 
a right answer to a problem, given sharply conflicting opinions. 
If that is the case, we then explore trade-offs between reason- 
ably right answers or in some cases least-bad outcomes. We are 
comfortable addressing these issues with more sophisticated 
versions of Charles's preference map for deciding where to live, 
or using game theory to achieve minmax solutions where you 
minimize the maximum loss involved. 


e Where values play an important role. Gun control in the US is an 
issue where values play an important role in shaping the debate 
about the issue. Sometimes these look intractable. Nonetheless, 
common ground is most likely to emerge from good problem 
solving about the sources of accidental and homicide deaths 
from gun ownership, and the potential for addressing some of 
these causes with relatively simple interventions (mental health 
and criminal record background checks, waiting periods, clos- 
ing sales loopholes). Death-penalty regulation may be a similar 
issue, where values are bridged when confronted with high 
error rates in death-penalty convictions. We have found that 
getting groups holding different values to participate in prob- 
lem disaggregation and fact gathering, can create bridges to 
common ground. 


e Where the real problem is nested inside other, more appar- 
ent problems. An example is homelessness, now an issue in 
many cities around the world. Few organizations addressing 
homelessness see shelter as the core issue, even though it has 
to be addressed. In addition to providing shelter, their efforts 
are directed at the underlying social, financial, and mental-health 
concerns and how they might be addressed. For example, a sig- 
nificant portion of women seeking shelter or refuge is related to 
domestic violence. The root cause of this mainly male behavior is 
complex to understand and address but is the key to long-term 
solutions. We are confident that the approach and tools we set 
out in the obesity case to arrive at cost-effective solutions are just 
as appropriate to addressing the issue of homelessness. 
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Our look at wicked problems gives us confidence that you can 
employ the seven-steps process to good effect and insight on even 
the most challenging problems. There is no reason to put problems 
in the too-hard basket. To be informed citizens and voters, we all 
need to be willing to put our problem solving creativity to work on 
these toughest societal problems. 


Chapter 9 Takeaways 


e Some problems are particularly difficult because they are part 
of complex systems, with multiple causes, many stakeholders, 
involve externalities, and require difficult behavioral change to 
affect—these have been labeled wicked problems in the policy 
literature over the past 40 years and include terrorism, climate 
change, homelessness, and obesity. 


e While these problems are harder to solve, we believe the same 
seven-steps framework can unlock insights into solutions. 


e Often the leverage on these most difficult problems comes from 
systems rather than partial solutions, from making externalities 
endogenous to the problem, and from novel ways of cleaving 
the problem. 


Problems to Try on Your Own 


e List all the problems you can think of that are complex systems 
with many actors and multiple causes. 


e Build a tree for understanding the causes of homelessness; take 
one branch and see if you can think of ways to make the exter- 
nalities internal to the problem. 


e Build a tree for drivers of terrorism in Western countries; can you 
think of new ways to take this problem apart that uncover novel 
solution paths? 


Chapter Ten 


Becoming a Great 
Problem Solver 


We started the book by outlining the need for creative problem 
solving capability in all walks of life. There was a time when problem 
solving was viewed as the domain of a few professions like science, 
engineering, and management consulting. With the accelerating 
changes of the twenty-first century, it is no longer a skill in limited 
domains but an expectation of individuals and teams across the 
business, non-profit, and government sectors. People are increas- 
ingly hired based on demonstrated analytic and thinking skills, 
evaluated for creative problem solving, and promoted for their abil- 
ity to mobilize agile teams to rapidly address changing demands. 
This is the problem solving century. 


We are convinced that good problem solvers are made not born. 
In the book we have taken a close look at what great problem solv- 
ing entails. Our goal has been to help you become confident and 
creative problem solvers, to feel competent and empowered to 
employ these techniques for problems at any scale. The seven-step 


253 


254 


Becoming a Great Problem Solver 


process we have outlined provides a way to take a business, 
personal, or societal problem you face and work it through to a 
solution. The 30 cases we have discussed cover the gamut of prob- 
lems you are likely to encounter in your work and life. We hope no 
mystery remains about what is involved in great problem solving. 


The seven-step bulletproof problem solving process doesn’t 
require a degree in higher mathematics or logic. Great problem 
solving consists of good questions that become sharp hypotheses, 
a logical approach to framing and disaggregating issues, strict pri- 
oritization to save time, solid team processes to foster creativity 
and fight bias, smart analytics that start with heuristics and move to 
the right big guns, and finally a commitment to synthesize findings 
and turn them into a story that galvanizes action. We are confident 
that with practice and experience you will cleave problems in new 
and clever ways, learn to separate root causes from symptoms, 
and stay on the critical path of efficient problem solving. We have 
this confidence because over our careers we have seen others 
go from being bright graduates to impressive problem solvers in 
a short time. 


There are some last words of encouragement and advice that we 
want to provide in this closing chapter. They are the watchwords of 
the seven-step process, what we feel so strongly about when we 
discuss this approach with our teams. There are 10 points in all to 
keep in mind. They are: 


1. Take the time up front to really understand your problem. 
Often the initial problem that is posed isn’t the right question. 
Take the time to probe the problem carefully with the decision 
makers involved before racing off to conduct analysis. Know 
the boundaries (it is often worth testing them to maximize crea- 
tivity), the accuracy required, the time frame allotted, and any 
other forces acting on the problem. Be ready to revise your 
problem statement as you learn, refining it iteratively with one- 
day answers. Remember, the whole idea is to motivate action 
for change. 
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2. Get started with nothing more than a problem statement. 
You don't need to wait for giant data sets or fancy computer 
models. Get a large piece of paper and a pencil, or better yet a 
whiteboard, and begin sketching out a logic tree for your prob- 
lem. When you dive into the data you can expect your thinking 
will evolve and move from a simple component structure to 
more sophisticated hypotheses. At this stage you are simply 
exploring what has to be true in order to support a hypothesis. 


3. Try several cuts at the tree. We often write components or 
branches on sticky notes and move them around until they 
make sense in a logical grouping. You can think of this as solv- 
ing the puzzle backward from the most tangible, known pieces 
you have. Try one or more cleaving frames to see which ones 
yield the most insight. Then lay out the main relationships that 
follow from the disaggregation you've selected, ideally in a 
mathematical fashion or one that exhausts possibilities. 


4. Use a team wherever you can. Teams bring great advantages 
to problem solving via diversity in thinking and experiences 
that deepen the richness of creativity and reduce the chance 
of confirmation and other persistent bias. If you are on your 
own, make a team of colleagues, friends, family, and neighbors. 
We created virtual teams of knowledgeable people to test our 
thinking and hypotheses for our case examples, especially 
ones that involve deep-domain expertise. This made us much 
smarter than we are on our own, and it also makes it fun. Try 
techniques like team voting to overcome authority bias, red- 
team/blue-team competitions to be aware of other perspec- 
tives, and mock trials to advance the antithesis. 


5. Make the right investment in a good workplan. A good work- 
plan takes a little upfront time but will save so much wasted 
effort later. Prune your tree savagely by focusing on the big 
levers of impact that you can move. Be piercingly precise about 
what a particular output should look like, what hypothesis it 
questions, who will do it and by when. Use chunky plans that 
run only two to four weeks out so you don’t over-run your initial 
thinking, and lean Gantt study plans to keep your work on track. 
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6. Start your analysis with summary statistics, heuristics, and 


rules of thumb to get a feel for the data and the solution 
space. Before you dive into giant data sets, machine learning, 
Monte Carlo simulations, or other big guns, we believe it is 
imperative to explore the data, learn its quality, understand 
the magnitudes and direction of key relationships, and assess 
whether you are trying to understand drivers to plan an inter- 
vention or predict a state of the world. Sophisticated analysis 
has its place, but it is our experience that one-day answers, 
supported by good logic and simple heuristics, are often suf- 
ficient to close the book on many problems, allowing you to 
move on to the more difficult ones. 


. Don't be afraid to employ big analytic guns when required. 


Sometimes a complex problem like bus routing across a major 
city, detecting disease in medical images, or optimizing pro- 
duction facilities for a global company, does require advanced 
analytics. Most of the tools you may have encountered in a sta- 
tistics or operations research course are now more accessible 
with simple, powerful, and intuitive software packages. You 
also have the option of outsourcing your big-gun problems 
to eager teams of crowd-sourced machine learning practition- 
ers who will analyze your data and make predictions for mod- 
est encouragement and occasional remuneration. When your 
objective involves other actors whose behavior will change 
when you make your moves, especially when played out over 
longer and more uncertain periods, it is time to invest some 
effort in game-theory models, risk-management actions, strat- 
egy staircases, and long-term theories of change. 


. Put as much effort into synthesis and telling the story as 


doing the analysis. It’s natural to declare a problem solved 
when you have completed a powerful analysis that reveals great 
insight about a problem. But most of the problem solving you 
will ever do in business, non-profits, or politics involves per- 
suading someone to do something differently. This is why the 
problem is rarely solved at the point of your Eureka moment. 
You'll have to be persuasive to convince powerful stakehold- 
ers to follow your plans. Remember, humans are visual learners 
and love storytelling above all else. 
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9. Treat the seven-steps process like an accordion. We have 
frequently referred to the seven steps as an iterative process. 
We have also sought to emphasize that you can compress or 
expand steps depending on the issue, so in some ways it is 
also like an accordion. Use one-day answers to motivate the 
team and decision maker to go to the level of analysis that the 
problem requires. 


10. Don't be intimidated by any problem you face. If you invest 
the time in mastering the seven-steps process we are con- 
fident you will feel ready to tackle almost any personal, busi- 
ness, or societal problem that comes your way, even the wicked 
ones. This isn’t hubris. It is simply saying that with a reasonable 
investment in systematic problem solving you will find insight 
into almost any problem of consequence. That is why bullet- 
proof problem solving feels like a superpower when you get 
good at it. 


We don't see the problems of our lives as individuals, workers, and 
citizens getting any easier. Most of our institutions haven't invested 
enough in creative problem solving to find clever solutions and 
bridge differences. Now we have that opportunity and we hope 
you'll do your part to contribute solutions to the challenges of 
our century. 
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LEVERS THAT IMPACT RETURN ON INVESTED CAPITAL (ROIC) 
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PRIORITIZATION 
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Prioritization Worksheet 
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WORKPLAN 
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Workplan Worksheet 
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STORYLINE TREE: ARGUMENT STRUCTURE 
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Reasoning by analogy, 119 
limitations, 119-120 
Reasoning toward implications, 
184-185 
Refined hypothesis tree, 53-58 
Regional strategies, 
management, 230-233 
Registered nurses (RNs), 
addition (benefits), 66 
Regression analysis, 136 
performance, 
improvement, 146 
Regression models 
case study, 143-146 
problem, 145 
Regression tools, 
usage, 138, 140 
Regression, usefulness, 145 
Regulate/incent (cleaving frame 
example), 77-78 
Relocation, 71 
Relocation decision (case 
study), 17-22 
brainstorming, 18e 
flowchart, 19e 
town ranking, 21e 
Reshaping/editing, 71 
Resolution. See Situation- 
observation-resolution 
Resource companies, reserves 
acquisition, 214 
Results, synthesis, 179 
Retirement savings survival 
actions, 211-213, 212e 
case study, 208-213 
risk preferences, 209e 


Return on invested capital 
(ROIC), 58-59 
analysis, 183e. See also 
Hechinger/Home Depot. 
levers, impact, 65e, 260e 
tree, 62 
Rhodes Trust, 68 
Ripper Company, shape- 
interpreting algorithm, 162 
Ripper Group 
shark-spotting algorithm, 224 
staircase, construction/ 
example, 223-226, 225e 
Risk tolerance, consideration, 
211 
Rittel, Horst W.J., 235 
RNs. See Registered nurses 
Roberts, Stephen, 137 
Robert Wood Johnson 
Foundation, 145, 240 
Role playing, 104 
Root cause 
example, 131e 
exploration, 130 
Root cause analysis (RCA), 
128-129, 129e 
Root-cause tool, 123 
Routine tasks, xvi 
R-Project, 136 
Rules of thumb, 112-123 
Runner's knees, injury decisions, 
125-126 


S 
Samandari, Hamid, xxii 
Say It with Charts (Zelazny), 182 


Scenarios, usage, 218 
Schleicher, Andreas, xvii 
School buses (routing), 

computers (usage), 161 
Scripting language, 


learning, 159 
Scully-Power, Paul, 224 
S-curve, 116 


Seed, change stage, 228 
Sensitivity analysis, 
157-158, 218 
Sexually transmitted infections 
(STIs), violence 
(correlation), 42 
Sham control, 126 
Shark spotting drones 
Ripper Company, impact, 
162 
usage, 162-164 
Shortcuts, 112, 114 
examples, 113e 
Sibony, Olivier, 101, 107, 123 
Signal and the Noise, The 
(Silver), 118 
Sihvonen, R., 125, 126 
Silver, Nate, 119 
Simon, Herb, xxvi 
Simulations, 140 
case study, 141, 157-159 
model, usage, 158 
usage, 157 
Situation-observation- 
resolution, 95, 189e 
example, 96-97 
order, 95-96 
story, 184 
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structure, 190 
usage, 184-185 
Skills, ranking (2020), xvi 
SKUs. See Stock keeping units 
Sleep apnea onset, computer 
prediction, 160-161 
SMART. See Specific, measurable, 
action oriented, relevant, 
and timely 
Smit, Sven, 201 
Social networks, impact, 241-242 
Social problems, root cause 
analysis (application), 130 
Solar panels (case study) 
costs, decline, 92 
criteria, sustaining, 13 
first cut, 14e 
installation, decision, 12—17 
logic tree, 14e 
problem disaggregation, 16e 
Soltas, Evan, 153, 156 
Sosic, Rok, 143 
Space Shuttle Challenger 
disaster, xx 
case study, 141, 146-149 
O-rings/temperature, 
damage, 147e, 148e 
Specific intervention, impact, 150 
Specific, measurable, action 
oriented, relevant, and 
timely (SMART), xxi, 33 
Specific observation, 58 
Staircases 
approach, 220 
architecture, 222-223 
drivers, 223e 
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Staircases (continued) 


construction, 223-226 
example, 221e 
growth staircases, 220 
origins, 220 

platform, 221e 

steps, 220 


Strategies, portfolio, 203 
long-term strategy portfolio, 
management (case 
study), 226-233 
map, 228-229, 229e 
Strategy Beyond the Hockey 
Stick (Bradley/Hirt/ 


Smit), 201 
Stretch, staircase architecture 
consideration, 222 
Stuckey, John, 76 
Sullivan, Josh, xv 
Summary statistics, usage, 
127, 137, 256 
Superforecasters, 165 
psychological profile, 109e 
Superforecasting (Tetlock/ 
Gardner), 165 
Supply/demand (cleaving 
frame example), 78, 79e 
Sutherland, Jeff, 108 
Sydney Airport (case study) 
breakdowns, 9e, 10e, 11e 
capacity, 8-12 


strategic staircases, 
usage, 220-226 
strategy approach, 203 
Start-up company pricing 
decision (case 
study), 22-25 
customer phone survey, 25 
flowchart, 24e 
profit lever tree, 23e 
Stata (statistical software), 156 
Statistical distributions, 105-106 
Statistical software, usage, 156 
Steel company investments, 
highest-level problem 
(defining), 39-41 
Step-fixed cost, 120 
STIs. See Sexually transmitted 


infections Synthesis, usage, 256 
Stock keeping units (SKUs), 62 
Storyline tree worksheet, 263 T 
Storytelling, 179, 181, Taleb, Nassim Nicolas, 
185-190, 257 68, 105, 112 


Team members 
diversity, 102-103 


storyline, 191e 
Storytelling with Data 


(Nussbaumer preparation, 90-91 
Knaflic), 182 Team processes 
Strategic options, acquisition, agile team processes, 

200 106-109 
Strategic staircases, building, 87 


usage, 220-226 example, 83e 


importance, 184 
usage, 82-84, 95, 97-99 
Teams 
behaviors, 99-100 
brainstorming practices, 
103-105 
brainstorming session, 53 
characteristics, 98-99 
distributed voting, 105 
hypothesis/data, porpoising, 
99 
norms, 98-99 
problem solving, 98-99 
process-design approaches, 
impact, 101 
structure, leadership 
(relationship), 97-98 
structure/norms, neglect, 
xXii—xXiii 
tradeoffs debate, 222 
usage, 255 
Technological innovations, 
adoption curves, 117e 
Tennis services, game 
theory, 171-176 
examples, 172e, 173e, 175e 
Tercek, Mark, 246 
Terminal-value estimate, 
usage, 218 
Tetlock, Philip, xxii, 100, 
103, 104, 165 
The Nature Conservancy (TNC), 
115, 245-246 
permits, 246-248 
The Nature Conservancy 
Australia 
(TNCA), focus, 81 
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Theory of change (TOC) 
example, 227e 
map, 227-228 
Thinking, Fast and Slow 
(Kahneman), 99 
Thinking, sequencing, 137-138 
Thinking Strategically (Nalebuff/ 
Dixit), 173, 175, 220 
Three-question rule, 127 
Threshold 
questions, 69e 
setting, 98 
Tie, H.T., 242 
Time frames 
case examples, 202-203 
usage, 195 
Timetable, building, 6 
T-Mobile, lawsuit, 171 
TOC. See Theory of change 
Todd, Peter M., 114, 124 
Topical strategies, management, 
230-233 
Toyota Motor Corporation (5 
Whys), 128, 131e 
Trees. See Logic trees 
branches, 55 
component tree, 53-58 
cuts, usage, 255 
deductive logic trees, 58-69 
inductive logic trees, 67-68 
preparation, 7 
profit lever tree, 23e 
reduction, 255 
refined hypothesis 
tree, 53-58 
usage, 103 
visual presentation, 55, 56e 
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Truckgear (hypothetical company), 
22-24, 56, 120, 214 

Type 1 fast thinking errors, 100 

Type 1 thinking, 99-100 

Type 2 thinking, 99-100 


U 
UAVs. See Unmanned 
aerial vehicles 
Uncertainty 
addressing, 199e, 216 
assessment, 197 
case examples, 202-203 
handling, 211 
levels, 196-198, 197e, 220 
logic trees, usage, 199-202 
scenarios, 216-219 
usage, 195 
Unicorn, creation, 118 
Unintuitive statistician, 202 
Unit price, knowledge, 120 
University of Technology 
Sydney (UTS), 162 
Unmanned aerial vehicles 
(UAVs), usage, 224 
US Department of Homeland 
Security, passenger- 
screening algorithm, 164 
User-centered approach, 44 
User experience, 43—44 
UTS. See University of 
Technology Sydney 


V 

Valuation, approach, 218 

Value, development options 
(inclusion), 214-216 


Variables, addition, 146 
Verizon, lawsuit, 171 
Viguerie, S. Patrick, 199 
Visual tree presentation, 
56e 
Voter prejudice 
case study, 141, 152-157 
presumption, 154 
testing, 155e 


W 
Wald, Jeff, 204 
Walk Economy, 243 
Waterman, Bob, 107 
Webb, Caroline, 103, 104 
Webber, Melvin M., 235 
Webb Pressler, Margaret, 63 
Weldon, Kevin, 224 
Wessels, David, 214 
What if competitive scenario 
analysis, 74 
“What you have to believe” 
analysis, 74 
White, David, 203, 220 
Whitman, Walt, 69 
Wicked problems, 235 
obesity, 236-243 
overfishing, 243-250 
WiFi intellectual property 
(CSIRO defense), 118, 
168-171, 170e 
Wolf Ditkoff, Susan, 130 
Working backwards, 
value, 223-224 
Workplan 
analyses, 181 
building, 6, 87 


chunkiness, 93, 94e 
detail, 92e 
evolution, 95 
execution, 185 
investment, 255 
model workplan, 91e 
worksheet, 262e 
Work planning 
best-practice 
approaches, 89-90 
example, 90e 
project management, 
relationship, 89-93 
team processes, usage, 95-97 
Work/play (cleaving frame 
example), 80 
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World Economic Forum 
Future of Jobs Report, xvi 
new jobs list, 204 

World Health 

Organization (WHO) 
Commission on Early 
Childhood Obesity, 
242 
disease impact measure, 
238 
World Wildlife Fund (WWF), 115 


Z 

Zelazny, Gene, 182 

Zillow, home value model, 165 
Zutavern, Angela, xv 
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